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Preface 



The present book includes a set of selected papers from the Sixth International 
Conference on Informatics in Control Automation and Robotics (ICINCO 2009), held 
in -Milan, Italy, from 2 to 5 July 2009. The conference was organized in three 
simultaneous tracks: Intelligent Control Systems and Optimization, Robotics and 
Automation, and Systems Modeling, Signal Processing and Control. The book is 
based on the same structure. 

ICINCO 2009 received 365 paper submissions, from 55 different countries in all 
continents. From these, after a blind review process, only 34 were accepted as full 
papers, of which 21 were selected for inclusion in this book, based on the 
classifications provided by the Program Committee. The selected papers reflect the 
interdisciplinary nature of the conference. The diversity of topics is an important 
feature of this conference, enabling an overall perception of several important 
scientific and technological trends. These high quality standards will be maintained 
and reinforced at ICINCO 2010, to be held in Funchal, Madeira - Portugal, and in 
future editions of this conference. 

Furthermore, ICINCO 2009 included 5 plenary keynote lectures given by Daniel S. 
Yeung (University of Technology, China), Maria P. Fanti (Polytechnic of Bari, Italy), 
Janan Zaytoon (University of Reims Champagne Ardennes, France), Alessandro Giua 
(Universita' di Cagliari, Italy) and Peter S. Sapaty (Institute of Mathematical 
Machines and Systems, National Academy of Sciences Ukraine). We would like to 
express our appreciation to all of them and in particular to those who took the time to 
contribute with a paper to this book. 

On behalf of the conference organizing committee, we would like to thank all 
participants. First of all to the authors, whose quality work is the essence of the 
conference and to the members of the Program Committee, who helped us with their 
expertise and diligence in reviewing the papers. As we all know, producing a 
conference requires the effort of many individuals. We wish to thank also all the 
members of our organizing committee, whose work and commitment were invaluable. 

July 20 1 Juan Andrade-Cetto 

Jean-Louis Ferrier 
Joaquim Filipe 
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Abstract. Intelligent Transportation Systems (ITS) modelling, planning, and 
control are research streams that, in the last years, have received a significant 
attention by the researcher and practitioner communities due not only to their 
economic impact, but also to the complexity of decisional, organizational, and 
management problems. Indeed, the increasing complexity of these systems and 
the availability of the modern Information and Communication Technologies 
(ICT) for the interaction among the different decision makers and for the 
acquisition of information by the decision makers, require both the development 
of suitable models and the solution of new decision problems. The paper is 
aimed at showing the new attractive researches and projects in the fields of ITS 
operational control and management by using effectively and efficiently the 
latest key ICT solutions. In particular, the paper focuses on the ICT applications 
on the urban traffic control and the intermodal transportation network planning 
and management. 

Keywords: Intelligent transportation systems, Urban traffic control, Intermodal 
transportation networks, Information and communication technologies. 



1 Introduction 

Intelligent Transportation Systems (ITS) refer to the application of Information and 
Communication Technologies (ICT) to transport infrastructure and / or to vehicles to 
improve the efficiency of transportation networks. In recent years, the European 
Union has sponsored several projects targeting advancements of different 
transportation systems. On the other hand, ITS topics are considered relevant and 
attractive research areas. In particular, research and projects on the ITS may be 
broadly divided into infrastructure projects and vehicle-orientated applications. 
Typical infrastructure projects and researches include the planning of freight and 
people transportation operations and the traffic management. Vehicle-orientated 
projects and researches include applications such as automated vehicle location and 
planning. This paper focuses on the infrastructure planning and management that 
apply ICT tools. The advent of ICT has a tremendous impact on the planning of 
freight and people transportation and on traffic management. Indeed, ICT increase the 
flow of available data, improve the timeliness and quality of information and offer the 

J. A. Cetto et al. (Eds.): Informatics in Control Automation and Robotics, LNEE 85, pp. 3-|_13j 
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4 M.P. Fanti 

possibility to control and coordinate operations and traffic in real-time. Significant 
research efforts are required to adequately model ITS, specify and characterize the 
various planning and management problems and to develop efficient solution methods 
based of the application of the ICT tools. The aim of the paper is presenting the last 
contributions in term of projects and research in the fields of the planning and 
management of ITS. Hence, the paper recalls in Section 2 the most important projects 
supported by the European Commission (EC) in the areas of ITS. Moreover, Sections 
3 and 4 present the research advances in two crucial sectors of ITS: the control of 
Urban Traffic Networks (UTN) and the management of Intermodal Transportation 
Networks (ITN), respectively. In the two cases the paper enlightens the benefits of the 
ICT solution applications and the relative open problems. Finally, Sections 5 draws 
the conclusions. 

2 European Projects in the Field of ITS 

The potential of ITS applications has encouraged the EC to refer to them in its 
2006 mid-term review of the transport white paper. In particular, the grant program 
for the Trans-European Network for Transport (TEN-T) [35] aims to establish 
interconnections, interoperability and continuity of services especially on long-distance 
routes and across borders. Its guidelines cover the infrastructure for traffic management, 
traveller information, emergency systems and electronic fee collection. Within the 
framework of the TEN-T program for the period 2007-2013 the Easy Way project [35] is 
selected. Such a project is driven by national road authorities and operators with 
partners including the automotive industry, telecom operators and public transport 
stakeholders. TEN-T program identifies the set of necessary ITS European services to 
deploy (traveller information, traffic management and freight and logistic services) and 
is an efficient platform that allows the European mobility stakeholders to achieve a 
coordinated and combined deployment of these pan-European services. 

In the last years, the EC selected several projects that focus in turn on the planning 
and management of motorway, railway, and maritime traffic [16]. In particular, a 
basic project on ITS is CESAR I & II (Co-operative European System for Advanced 
Information Redistribution) that proposes an Internet communication platform that 
aims to integrate services and data for unaccompanied traffic and the rolling 
motorway traffic management. Moreover, in the field of railway system management, 
CroBIT (Cross Border Information Technology) is a new system that provides the 
railways with a tool to track consignments and integrates freight railways along a 
transport corridor providing total shipment visibility. A maritime navigation 
information structure in European waters is established by MarNIS (Maritime 
Navigation Information Systems) that is an integrated project aiming to develop tools 
that can be used to exchange maritime navigation information and to improve safety, 
security and efficiency of maritime traffic. 

In addition, several projects focus specifically on efficient freight transportation. 
For instance, Freightwise aims to establish a framework for efficient co-modal freight 
transport on the Norwegian ARKTRANS system. One of the main objectives in 
Freightwise is establishing a framework for efficient co-modal freight transport and 
simplifying the interaction among stakeholders during planning, execution and 
completion of transport operations. Moreover, the project e-Freight is a continuation 
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of Freightwise to promote efficient and simplified solutions in support of cooperation, 
interoperability and consistency in the European transport system. E-Freight is to 
support the freight logistics action plan, which focuses on quality and efficiency for 
the movement of goods, as well as on ensuring that freight-related information travels 
easily among modes. Furthermore, in the 7th Framework Program (FP7), the 
SMARTFREIGHT project wants to make urban freight transport more efficient, 
environmentally friendly and safe by answering to challenges related to traffic 
management and the relative coordination. Indeed, freight distribution management in 
city centres is usually operated by several commercial companies and there is no 
coordination of these activities in a way that would benefit the city. The main aim of 
SMARTFREIGHT is therefore to specify, implement and evaluate ICT solutions that 
integrate urban traffic management systems with the management of freight and 
logistics in urban areas. In addition, EURIDICE (European Inter-Disciplinary 
Research on Intelligent Cargo for Efficient, Safe and Environment-friendly Logistics) 
is a project sponsored by the EC under the FP7 seeking to develop an advanced 
European logistics system around the concept of 'intelligent cargo'. The goal is 
networking cargo objects like packages, vehicles and containers to provide 
information services whenever required along the transport chain. The project aims to 
build an information service platform centered on the individual cargo item and its 
interaction with the surrounding environment and the user. 

Moreover, a set of European projects focuses on the people transportation 
management. For instance the CESARE project (Common Electronic Fee Collection 
System for a Road Tolling European Service) started more than ten years ago and its 
development was structured into four project phases that are mainly focused on 
studying the technical aspects of the European interoperability. In particular, the 
CESARE IVth and final phase is specifying, designing, developing, promoting and 
implementing a common interoperability of electronic road toll systems in the 
European community, in order to function in a coordinated way. Furthermore, in the 
project called eMOTION public authorities, transport and telecommunication 
operators, IT suppliers and transport consultancies specify and assess multi-modal, 
on-trip traffic and travel information services for the European travellers. In 
particular, the range of services comprises real time traffic information for road traffic 
and public transport, dynamic (and multimodal) routing services and special services 
like reservation of parking space, booking of personalized public transport services as 
well as tourist information or hotel reservation. 

Summing up, the EC recognizes that innovation and ITS help to make transport 
more sustainable, which means efficient, clean, safe and seamless. In other word, new 
infrastructure cannot solve all transport problems of congestion and emissions and the 
integration of existing technologies can create new services. Consequently, for the 
next years the EC prioritizes projects involving application of ICT on transportation 
systems, in close collaboration with national governments, to ensure effective 
European coordination. 

3 Intelligent Transportation in Urban Traffic Control 

Traffic congestion of urban roads undermines mobility in major cities. Traditionally, 
the congestion problem on surface streets was dealt by adding more lanes and new 
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links to the existing Urban Traffic Networks (UTN). Since such a solution can no 
longer be considered for limited availability of space in urban centres, greater 
emphasis is nowadays placed on traffic management through the implementation and 
operation of ICT. In particular, traffic signal control on surface street networks plays a 
central role in traffic management. Despite the large research efforts on the topic, the 
problem of urban intersection congestion remains an open issue [10], [11], [23], [27]. 
Most of the currently implemented traffic control systems may be grouped into two 
principal classes [28]: i) fixed time strategies, that are derived off-line by use of 
optimization codes based on historical traffic data; ii) vehicle actuated strategies, that 
perform an on-line optimization and synchronization of the signal timing plans and 
make use of real time measurements. While the fixed time strategies do not use 
information on the actual traffic situation, the second actuated control class can be 
viewed as a traffic-responsive control policy employing signal timing plans that 
respond automatically to traffic conditions. In a real time control strategy, detectors 
located on the intersection approaches monitor traffic conditions and feed information 
on the actual system state to the real time controller, which selects the duration of the 
phases in the signal timing plan in order to optimize an objective function. Although 
the corresponding optimal control problem may readily be formulated, its real time 
solution and realization in a control loop has to face several difficulties such as the 
size and the combinatorial nature of the optimization problem, the measurements of 
traffic conditions and the presence of unpredictable disturbances. The first and most 
notable vehicle actuated techniques are the British SCOOT [20] and the Australian 
SCATS [24], that decides an incremental change of splits, offsets and cycle times 
based on real time measurements. However, although SCOOT and SCATS exhibit 
centralized hardware architecture, the strategies are functionally decentralized with 
regard to splits setting. A formulation of the traffic signal network optimization 
strategy is presented in [23] and [30]. However, the resulting procedures lead to 
complex mixed integer linear programming problems that are computationally 
intensive and the formulation for real networks requires heuristics for solutions. 
Indeed, the most available signal control strategies are not suitable for saturated traffic 
conditions because they fail to consider the downstream traffic conditions in the real- 
time decision making at individual junctions. Due to this reason, there is a lack of 
efficient and systematic co-ordinated control strategies applicable to large scale 
network. In order to solve such a problem, some contributions are proposed based on 
the so-called store- and-forward modelling approach [19] that describes the network 
traffic flow process so as to circumvent the inclusion of binary variables. This choice 
permits the use of efficient optimization and control methods. In particular, we recall 
the design approach of Diakaki et al. [9] that propose a traffic responsive co-ordinated 
urban control strategy formulized as an optimal control problem based on a store-and- 
forward modelling approach. More precisely, the urban traffic control problem is 
formulated as a linear-quadratic (LQ) optimal control problem leading to a highly 
efficient and extremely simple co-ordinated strategy applicable to large-scale 
networks. Moreover, in [1] the authors compare the methodology based on the linear 
multivariable feedback regulator derived through the formulation of the LQ optimal 
control problem with two novel methodologies based on the store-and-forward 
modelling paradigm. The authors highlight the advantages and the shortcomings of 
the different strategies. 
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Based on the same model, a recent paper [8] presents a prototype of computational 
platform to control an urban traffic network. The platform automatically implements a 
distributed Model Predictive Control (MPC) from the mathematical specification of 
the problem and runs a distributed algorithm. The platform prototype is applied in the 
green-time control of a real-world network and the distributed algorithm is compared 
with the LQ approach application from [1]. The simulation results show that the 
MPC-based approach outperform the LQ-based approach in term of traffic 
performance metrics. Moreover, the advantage of the distributed model predictive 
control approach is the easiness of expansion and reconfiguration of the control 
structure. 

Despite the efficiency of the proposed control strategies, such a modelling 
approach can not directly consider the effects of offset for consecutive junctions and 
the time- variance of the turning rates and the saturation flows. An improvement on 
urban traffic actuated control strategies is provided in [13] where the green splits for a 
fixed cycle time are determined in real time, in order to minimize the number of 
vehicles in queue in the considered signalized area. To this aim, the paper pursues 
simplicity in the modelling and in the optimization procedure by presenting a 
macroscopic model to describe the urban traffic network. Describing the system by a 
discrete time model with the sampling time equal to the cycle, the timing plan is 
obtained on the basis of the real traffic knowledge and the traffic measurements in a 
prefixed set of cycles. The traffic urban control strategy is performed by solving a 
mathematical programming problem that minimizes the number of vehicles in the 
considered urban area. The minimization of the objective function is subject to linear 
constraints derived from the intersection topology, the fixed cycle duration and the 
minimum and maximum duration of the phases commonly adopted in practice. 

Despite the large literature in the field of the urban traffic control, the future 
research has to give new contributions in facing the apparently insurmountable 
difficulties [28] in the real time solution and realization of the control loop governing 
an urban intersection by traffic lights. Future work will be devoted to provide 
methodologies enjoying the following properties: i) guaranteeing a real time network 
wide signal control in large scale urban traffic; ii) exhibiting an efficient control 
behaviour in different traffic scenarios, such as saturated traffic conditions; iii) 
developing additional algorithms for considering different real components such as 
bus-priorities and pedestrians; iv) optimizing the effects of offset for consecutive 
junctions. Finally, in order to validate the proposed methodologies, field applications 
to extended network urban parts are necessary. 

4 Intelligent Transportation in Intermodal Transportation 
Networks 

Intermodal Transportation Networks (ITN) are transportation systems integrating 
different transportation services, designed to move goods from origin to destination in 
a timely manner and using multiple modes of transportation [4]. The ITN 
management and planning are currently relevant subjects of research because ITN 
allow more efficient, cost effective and sustainable transportation than the traditional 
transportation systems [6]. However, ITN decision making is a very complex process, 
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due to the dynamical and large scale nature of these systems, the hierarchical structure 
of decisions, the multiplicity of actors involved, as well as the randomness of various 
inputs and operations. A systematic way to capture all decisions in the management of 
ITN typically proposed in the related literature is based on a three-level hierarchy: 
strategic, tactical and operational ones. Strategic level planning involves ITN design 
and considers time horizons of a few years, requiring approximate and aggregate data. 
Tactical level planning basically refers to the optimization of the flow of goods and 
services through a given logistics network. Operational level management is a short- 
range planning, involving transportation scheduling of all transporters on an hour-to- 
hour basis, subject to the changing market conditions as well as to unforeseen 
transportation requests and accidents. 

With the development of ICT, operative and tactical issues can be dealt in a 
different way than in the past, taking advantage of the effective impact of these 
innovative technologies on ITN decision making. Indeed, ICT solutions can increase 
the data flow and the information quality while allowing real-time data exchange in 
intelligent transportation systems and traffic networks [6], [13]. As discussed by 
Giannopoulos [20], numerous novel applications of ICT to the transportation field are 
currently in various stages of development, but in the information transfer area the 
new systems seem to be unimodal. In the application of ICT solutions to multimodal 
chains, an important and largely unexplored research field is the assessment of the 
impact of the new ICT tools on management and control of ITS at the operational 
level before their implementation, by a cost-benefit analysis [38]. In particular, an 
effective ITN model at the operational level should focus on evaluating performance 
indices describing activities, resources (cost, utilization and inventory), output 
(throughput, lead time) and flexibility (lead-time, lead time variability) by integrating 
information flows. 

In the related literature, simulation has represented an effective and useful 
instrument to analyze transport logistics and evaluate the impact of the proposed 
solutions [5], [34] [37]. However, ITN and their decision making processes exhibit a 
high degree of concurrency and are characterized by resource sharing and conflicts. 
Hence, ITN can be successfully modelled as Discrete Event Systems (DES), whose 
dynamics depends on the interaction of discrete events, such as demands, departures 
and arrivals of means of transportation at terminals and acquisitions and releases of 
resources by vehicles. DES models are widely used to describe decision making and 
operational processes in logistics systems. In particular, Ramstedt and Woxenius [32] 
analyze the literature about the simulation of the decision-making process within a 
transportation chain and Gambardella et al. [18] simulate a resource allocation 
problem in an intermodal container terminal. In a subsequent work, Rizzoli et al. [33] 
present a discrete-event simulation model of the flow of intermodal terminal units 
among and within inland intermodal terminals, serving a user area via a road network 
and interconnected by rail corridors. Moreover, Legato and Mazza [22] propose a 
queuing network model of the logistic activities related to the basic processes of 
vessels in a container terminal. Also in [29] a discrete event simulation model is used 
to analyze in an intermodal container environment the impact of new road and railway 
networks on the logistics system. Among the available DES models, Petri Nets (PN) 
may be selected as a graphical and mathematical technique, suitable to describe 
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concurrency and synchronization [31]. Indeed, processes in intermodal terminals are 
addressed in [17] with stochastic PN and in [7], [12], [14] with Timed Petri Nets 
(TPN). 

Since the cited models are designed to describe a particular ITN, it is evident in the 
related literature the lack of a systematic and general methodology to describe in 
detail the multiplicity of elements that can influence the ITN dynamics and the 
corresponding information and decision making structure. An effort to propose a 
methodology able to thoroughly and systematically model a generic ITN is performed 
by Dotoli et al. [14] that use a modular modeling approach in a TPN framework to 
describe and simulate the ITN behavior. Moreover, in [14] the application of ICT 
solutions that allow sharing information among stakeholders in an intermodal system 
is analyzed and some performance indices are singled out to evaluate activities, 
resources (utilization) and output (throughput, lead time). A case study is modelled 
and simulated considering an ITN constituted by a port and a truck terminal of an 
Italian town including the road-ship transhipment process. The simulation results 
show that ICT have a huge potential for efficient real time management and operation 
of ITN, as well as an effective impact on the infrastructures, reducing both the 
utilization of the system resources as well as the cost performance indices. 

In addition, paper [2] is a contribution to propose not only a systematic modelling 
approach devoted to describe a generic ITN, but also a management technique based 
on a closed loop mode. To this aim, papers [2] develops an Integrated System (IS) to 
be used by decision makers that have to take operational as well as tactical decisions 
in large and complex ITN and may rely on information based services. The proposed 
management and planning approach is based on the specification of two main 
modules that are the core of the IS: an ITN reference model and a simulation module. 
In particular, the reference model describes systematically and in detail a generic ITN, 
providing the system states to the simulation module that foresees the evolution of the 
ITN. In order to obtain a generic model describing a non-specific ITN, the paper 
proposes a metamodelling technique that applies to models. Moreover, the 
metamodelling approach provides an accurate description of the constructs and rules 
needed to obtain semantic models that encapsulate all the concepts necessary to 
describe the structure and the behaviour of a particular system. More in detail, the 
considered metamodelling approach is a top down procedure based on the Unified 
Modelling Language (UML) [25]: a graphic and textual modelling language intended 
to understand and describe systems from various viewpoints. In addition, the 
simulation module is specified by identifying an instance of the metamodel and by 
easily translating such UML formalism in the simulation software. Hence, the 
simulator reproduces and foresees the ITN behaviour and provides the Decision 
Making System (DMS) with suitable performance indices in order to evaluate the 
proposed decision strategies. 

The presented IS can work in two alternative ways, respectively devoted to the off- 
line planning (tactical level) and the on-line management (operational level) of the 
ITN. Figure 1 shows the IS structure for the tactical level decisions: the IS bases on 
the reference model analysis and simulation the detection of the system anomalies and 
bottlenecks. Successively, the IS proposes, tests and evaluates some solutions on the 
basis of the estimation of suitable performance indices. 
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Fig. 1. The IS structure for the operational level application 



Fig. 2. The IS structure at the tactical level application 

Figure 2 depicts the IS structure for operational decisions. In this case, the IS 
works in real time and the reference model is updated on the basis of data exchanged 
by the stakeholders and the information obtained by using modern ICT tools. 
Moreover, the simulation module reproduces and foresees the behaviour of the ITN, 
starting from the model state knowledge and the detected real events. Hence, the 
reference model supplies the decision making system with the knowledge base 
necessary for decisions in real time and for optimizing suitable performance indices. 
In order to show the effectiveness of the presented management approach, the paper 
[3] specifies and analyzes at the tactical level for a real ITN composed of the port of 
Trieste (Italy) and the inland terminal located in Gorizia (Italy). The case study is 
analyzed in the framework of the EURIDICE Integrated Project, sponsored by the EC 
under the PF7 [26] and the performances obtained by the simulation studies show 
how integrating ICT into the system leads to a more efficient system management and 
drastically reduces the system lead times. 

The papers [2] and [3] are contributions devoted to propose a systematic modelling 
approach to describe a generic ITN as well as to specify decision making strategies. 
Future research will address the open problems of the detailed description of all the 
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ITN nodes such as airports and railway systems, and the specification and design of 
the Decision and Management System modules. 

5 Conclusions 

The paper presents some new attractive researches and projects in the field of ITS 
planning, control and management. In particular, the key solutions of using 
effectively and efficiently the latest developments of ICT for ITS management are 
pointed out. To this aim, the paper recalls the most important European Projects that 
are selected to make freight and people transportation more sustainable, efficient and 
safe by exploiting novel ICT solutions. Moreover, two crucial fields of the ITS 
management and control are considered: the control of Urban Traffic networks and 
the management of Intermodal Transportation Networks. In the two cases are 
emphasized the results and the advances of the ICT applications and the challenges of 
future researches. 
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Abstract. Safety verification for hybrid systems is a very important issue but 
also a very complex one. The abstraction is a mean to simplify it by shifting 
from global considerations to local ones. The aim of this paper is to present an 
overview of techniques of abstraction of hybrid systems that are commonly used 
for safety analysis. Reachability approaches related to abstraction for verification 
are briefly presented. 

1 Introduction 

Formal verification of properties is a very important area of analysis of hybrid dynami- 
cal systems. It is, indeed, essential to use methods and tools to guaranty that the global 
behaviour of the system is correct and consistent with the specifications. This is espe- 
cially true for safety properties that insure that the system is not dangerous for itself or 
its environment as long as the assumptions on which the model is built are fulfilled Q). 

Safety verification is a complex task and one approach to deal with it consists in 
defining a new model that is simpler to analyse in such a way that the conclusion on the 
simple model is guarantied to be valid for the initial model. As behaviour of systems is 
concerned and safety guaranty is the aim, this simplification is based on the concept of 
abstraction. Of course a part of the work load that is no longer required for the global 
analysis has to be spent on the abstraction process, but the considerations are then local 
and therefore easier to deal with. 

The aim of this paper is to give an overview of the concept of abstraction for safety 
analysis and the related reachability approaches. In Sect. |2]the formal model of hybrid 
automaton is presented and the concept of abstraction is specified. Then two types of 
abstractions are considered. In the first one, presented in Sect. the hybrid nature of 
the system is retained but its continuous dynamics is simplified by linear differential 
inclusions. The second one (Sect.Q considers a purely discrete event system to abstract 
the system. As it will be seen in these sections, reachability is a central issue in the 
abstraction process and therefore some issues on reachability are finally presented in 
Sect. [5] A more detailed presentation of reachability may be found in [2|. 

2 Hybrid Systems and Hybrid Automata 

2.1 Hybrid Automata 

Hybrid systems are characterized by sequential and continuous dynamics that interact 
and, therefore, require specific modeling formalism to represent these phenomena. Such 
a formalism can be based on hybrid automata inspired from [3 1. 
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A hybrid automaton, without synchronising events is specified by the tuple 
< L, X, U, Inv, F, E, Guard, Jump > where: 

- L is the set of locations or discrete states, X is the continuous state space and U is 
the space of continuous inputs, 

- Inv is a function that maps each location to a region of X x U: Inv(k) or, what will 
be considered as equivalent here, is a set of predicates on state and input variables 
that characterizes this region, 

- F associates to each location U in L, the continuous dynamics F(li,x, u), 

- E is a set of discrete transitions where each transition e is associated with a pair of 
locations (I, I') 

- guard is a function that associates to each transition a region in X x U or a set of 
predicates on state and input variables 

- Jump is a function that associates to each transition a map from the state and input 
spaces to the state space. 

The behavior of such an automaton is specified by the set of admissible trajectories 
from an initial state. Each element, r, of the set ((?,, Xj), u(i)) of trajectories from a 
given state (k, Xj) subject to the input signal u(i), is a finite or infinite ordered set: 

r = {(l To , 0, # ), (In , tl, *i), ' • • , (lr k , tfc, *fc), ■ ■ •} 
such as 

- l Ta = h and<P (0) =Xj, 

- for all indices k, Vi G [ifc, *fc+i [: the continuous state is x(t) = &k(t), the continu- 
ous dynamics is specified by the activity of the location 4>k (t) G F(/ Tfc ,x(t),u(t)) 
and the continuous state remains within the invariant of the location (<Pk (t), u(i)) G 
Inv(l T J 

- for all indices fc, there exists a transition e = (I, I') in S such as: I = l Tk ,l' = l Tk+1 < 
the state before the firing is within the guard (<Pk(tk+i), u(ife+i)) G guard(e), and 
the initial state in the next location is the result of applying the jump function to the 
state before the firing <P k+1 (t k+1 ) = Jump(e) ((<P fe (i fe+ i),u (tfc+i))). 

For a given trajectory, each term (l Tk ,tk 1 <I>k) is then associated with a time interval 
[tfe, tfe+i] during which the location does not change and the continuous state evolves 
according to the continuous dynamics associated with this active location l Tk , thus 
defining a continuous transition. Each time tk is a discrete transition firing time when 
the location changes. A trajectory is then a sequence of discrete and continuous 
transitions. 

For a trajectory r of a hybrid automaton, the state at time t will be denoted r(i) and 
is defined by r(i) = (l Tk , &k(t)) where k corresponds to t G [tk, ifc+i [• 

These definitions of a hybrid automaton and of the set of its trajectories from one 
hybrid state, leads to the following definitions of the successor sets of a point of the 
hybrid state (Z,,Xj). 

The hybrid successor set of (U, Xj) is the set of states that are reachable by a trajectory 
starting in (£j,Xj). 

Succ H {{k,Xi)) = {(/ fe ,x)|3u(t) s.t. 3t g 0{{k,Xi),u(t)), 3t, r(t) = (Z fe ,x)} 
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The discrete successor set of the point (k,Xi) is the set of points reachable by the firing 
of a discrete transition: 

Succ D ((li,Xi)) = {(Z fc ,x)|3e = (k,h) E E, 3u£ U s.t.(*i,u) £ Inv(k) 
A((x i; u) G guard(e)) A (x = Jump(e)((xi,u))} 

The continuous successor set of the point (li,Xi) is the set of points reachable by the 
firing of a continuous transition: 

Succcdk^x,)) = {(k,x)\3u(t) s.t. 3t e 6>((Z,,x 4 ),u(i)), 3t e [0,ti], x = $ (t)} 

Symmetrically from these definition of successor sets it is possible to define the prede- 
cessor sets of (h,Xi) as the states from which it can be reacheco 

PredX(U,*i)) = {{l,x)\(k,xi) e Succ.((l,x))} 

These predecessor sets can be explicitely specified and for example the discrete prede- 
cessor set is given by: 

Pred D ((k,Xi)) = {(l k ,x)\3e = (l k ,l t ) e E, 3ue U s.t.(x,u) E Inv(l k ) 
A((x, u) G guard(e)) A (x.; = Jump(e)((x, u))} 

These definitions are easily extended to regions of the hybrid state space considering 
that the image of the region is the union of the images of its points. For example, the 
continuous and discrete successor set of a region R are given by: 

Succc(R) = [J Succc (Oi,x)) , Succ D (R) — [J Succ D ((h,*)) 

(k,x)£R (k,x)£R 

2.2 Abstraction 

Abstraction is a concept that characterises the relationships between two transition sys- 
tems. It is based on the existence of a relation from the state space of one system to the 
state space of the other one, such as the image of each trajectory of the first system is 
a trajectory of the second one. This notion may easily be related to reachable spaces 
of the two transitions systems, and expressed by (Q~|l, where tt denotes the map that 
characterizes the abstraction, Succi the reachable space according to the first transition 
system, Succ-i the reachable space according to the second transition system, and \ a 
state of the first system: 

tt (Succi(x)) C Succ 2 (tt(x)) (1) 

This is usefull when considering safety properties of systems that can be expressed as 
the unreachability of specific regions of the state space from an initial region. Then the 
computation of the reachable space of the abstract system may be sufficient to prove 
the property for the first system provided that the mapping preserves the expression of 
the property. 



In this expression the . symbol denotes H, C, or D. 
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Fig. 2. Reachability for linear inclusion 



3 Linear Differential Inclusion Abstraction 

A first level of abstraction that may be useful to consider when dealing with verification 
and reachability is the abstraction of the continuous dynamics. 



3.1 General Principle 

The general principle of this approach [4| is illustrated on Fig.Q] As each location I 
of the hybrid system is associated with an invariant region, it is possible to compute 
the image of this region by the flow function F(li,x, u) associated with the location. 
This image (or one of its over-approximations) can then be considered as specifying the 
continuous dynamics of the abstracted location by differential inclusion. 

So if we consider a hybrid automaton AH =< L, X, U, Inv, F, E, G, J >, the au- 
tomaton defined by the relations below is an abstraction where the abstraction mapping 
7r is the identity function. 

- L* = L, X* = X, U* = U, Inv*(h) = Inv(k) 

- F*(l,x,u) = Fi^>F(l,Inv(l)) 

- E* = E,G* = G,J* = J 

If the set Fi D F(l, Inv(l)) is specified by affine constraints w T x < b, it is possible 
to easily compute the continuous reachable space by considering the most restricting 
constraints w T x < that will define with the initial region the borders of the reachable 
space. For example on Fig. [2] the linear inclusion (Fig.[2]a) allows computing the most 
restricting constraints specified by 71 and 72 that are used to compute the reachable 
space from P (Fig. 0b). 

Generally the abstraction that is obtained by this method is too coarse and the com- 
puted reachable space does not make it possible to conclude for safety verification. The 
solution is then to refine the abstraction by considering subsets of the continuous re- 
gions. From the specification of the original automaton AH and an abstracted automa- 
ton AH obtained with the previous method or after some refinement steps, a refinement 
step with respect to a location lj of the abstracted automata AH allows one to compute 
the next abstraction AH and is specified by the algorithmQ] 



Abstractions of Hybrid Systems for Verification 19 



Algorithm 1. Hybrid abstraction refinement 



Given: AH a hybrid abstraction of AH the hybrid system ,7f the associated abstraction mapping, 

lj one location of AH such as (lj , Inv(lj)) — n(l, Inv(lj)) 
Result: AH the refined abstraction of AH and 7r the associated abstraction mapping. 
1: Init: AH <*= AH, tt <= tt 

2: choose a set of regions Dk such as UD* = Inv(lj) 

3: delete the location lj and the related elements from AH, define for all regions Dk, 
(lk,Dk) = n(l,Dk), add the new locations Ik to L and define for each new location 
Tnu(l h ) = D k . 
4: for each location Ik define the continous dynamics F(lk , x, u) = Fk D -F(Z, -Dfc ) 
5: for all output transitions of lj in AH, e = (/_, , Z p ) do 

6: if Guard(e) flBt/l then 

7: add the transition e^ = (Ik, l p ) to .E 

8: define Guard(ek) — Guard(e) n -Dfc and Jump(ek) — Jump(e) 

9: end if 
10: end for 

11: for all pairs (Ik, In) of states that were created at the substep[3]above do 
12: if a continuous trajectory goes from Dk to D n according to the initial dynamics 
F(l,x,(u)) then 



add the transition e — (Ik, In) to E 

Define Jump(e) as the identity and Guard(e) as the border of Dk 
end if 
end for 



This refinement procedure may be iterated and may lead to more precise abstraction 
but there is no guaranty that it will eventually make it possible to conclude. 

3.2 Affine Systems 

An important point in this abstraction procedure is the choice of the continuous domains 
that are used to define the abstraction mapping because it may induce difficulties to 
compute the image by the continuous dynamics F and the set of transitions between the 
new locations. However it is possible for classes of systems to use some properties of 
the dynamics to guide the choice of the domains. For example for affine systems, that is 
systems such as the continuous dynamics in each location are specified by x = A.x + b, 
it is possible (see [5| for more details) to consider families of hyperplanes defined by 
equation H = {x|q T x = k} where there exists a vector 7 such as q = A 7 and 
k = —j T b, because the differential inclusion is then defined by this vector 7. Moreover 
if the left eigenvectors of the matrix A are used to generate the family of hyperplanes, all 
trajectories cross the hyperplane in the same direction leading to a very simple transition 
structure for the abstraction. This is exemplified on Fig.|3]in the case where the vectors 
q are defined as linear combinations of complex left eigenvectors. This approach may 
be extended to systems where the dynamics are defined by x = A.x + Bu and where U 
the space of continuous inputs is a polytope [6|. 
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a b) 
Fig. 3. Linear abstraction for affine systems 



4 Discrete Event Abstraction 



The second type of abstraction of hybrid systems that we will consider is discrete event 
abstraction. While considering a hybrid system modelled by a hybrid automaton as 
specified by Sect. [2 the idea is then to build a discrete event system and a map between 
the two state spaces such as the relation (Q]) is satisfied. The concept of such discrete 
abstractions was introduced for theoritical study of the decidability of reachability com- 
putations for hybrid systems (see e.g. the paper by |j7)) but also to perform verification 

na. 



4.1 General Principles 

Below we will consider the problem of computing whether the forbidden region Rf of 
the state space is reachable from an initial region Ri for a hybrid automaton H using 
a discrete event abstraction. For simplicity reasons we will also assume that each of 
the regions Rf and Ri is associated with only one location that is Rf = (If, Df) and 
Ri = (li,Di). The question is then to determine whether: 



Rf fl Succn{Ri 



(2) 



The first step is to determine the set of states Q of the abstract system. As the aim is to 
prove that (O holds it is natural to first create two states q h and qf and to consider that 
qi = 7T ((li, Di)) and qf — n ((lf,Df)). The problem to solve will then change and the 
new problem will be to prove for the discrete system, that (O holds provided that the 
discrete system is an abstraction of the hybrid one. 



qf £ Succ(qi) 



(3) 



Several choices are possible for the other discrete states. The simplest one is to cre- 
ate a discrete state for each location such as qt — n ((Ik, Ink)) where Ink is the 
projection of the invariant on the continuous state space, i.e. Ink — { x |3u <E U s.t. 
(x, u) £ Inv(lk)}- This choice is however often too rough and it is often better to cre- 
ate a discrete state for each transition of the hybrid automaton with q e = tt (Ik, G e ) 
where Ik is the source of the transition e and G e is the projection of the guard on the 
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Fig. 4. Discrete mapping for abstraction on guards 





Fig. 5. Discrete mapping for abstraction on borders 

state spacq3 For example, on Fig. [4] each of the regions G\, G2, G3 (representing the 
projection of the guards of some transitions) and Di (representing the initial region) is 
associated with a discrete state, whereas D2 (representing the image of G\ by the jump 
function) is not. The transition from q\ to q% stems from the continuous reachability of 
G3 from Di- Another possible choice is to consider the borders of this set G e for each 
transition in order to get sets of lower dimension. For example, on Fig. [5] borders bjk of 
the previous regions are associated with discrete states qjk and the transition from qi to 
qn (but not to qn) expresses the fact that all trajectories from Di attain G\ through the 
border 612, whereas the transition from qn to 531 (but not from qi2 to 531) expresses 
the fact that G3 is reachable from bn but not from 612. 

When the set Q of states is defined, the second step consists in computing the transi- 
tions of the discrete model. The constraint to be satisfied is then: 



(Zfe.Xfc) G SUCC H ((ln,Xn)) =*► 7r((Zfc,Xfc)) G Succ (?T ((Z n , x n ))) 



(4) 



Relative to the principle used to define the discrete states, local considerations may be 
used to satisfy this constraint. For the examples given above it is possible to show that 
the constraint is satisfied when using the following procedure: 

- for qi — tt ((li, Di)) add a transition (qi,q) for all q = 7r ((Zj, D)) G Q such as 
(k,D)nSucc c (h,D i ))^9, 

- for all discrete states q^ G Q — {qi,q/} add a transition (qk,q) for all q = 
tt ((I, D)) e Q such as (I, D) n Succ c (Succ D (Succc (l k ,D k )))) / 0. 

When the discrete states are defined by the guards (case of Fig.|4| an alternative that is 
based on only one continuous successor set is given below. Of course this new procedure 
leads to another discrete event system that also satisfies constraint lO. 



i.e. G e — {x|3u G U s.t. (x, u) G Inv(lh) H Guard(e)}. 



22 H. Gueguen and J. Zaytoon 




q 



q 



Fig. 6. Spurious transitions 

- for q k = 7r ((lk,Dk)) add a transition (qk,q) for all q = n ((Ik, D)) G Q such as 
(l k ,D)nSuccc(lk,D k ))^9, 

- for all discrete states qk G Q — {q/} add a transition (qk, q) for all q = n ((I, D)) G 
Q such as (I, D) n Swcc c (Succd (Z fc , D fc ))) 7^ 0. 

During the abstraction procedure that leads to the discrete event system, the only con- 
sideration is to garanty that the satisfaction of (0 will prove the satisfaction of (0. 
However the non satisfaction of (0 does not necessary imply that is not satisfied. 
This is due to the over-approximation of the reachable space introduced by the abstrac- 
tion. This is illustrated in Fig.[6]where the abstraction process will lead to conclude that 
D4 is reachable from D\. 

If the abstraction is too rough to conclude about the satisfaction of (0 from 01 it is 
possible to refine it. The idea of refinement is to consider a discrete transition and to 
make a partition of the continuous domain of the region mapped to the source location 
in order to introduce two new discrete states, one from which the transition can be 
fired and the other one from which it can not ITOl . The detailed process depends on the 
process used to build the abstraction. For example if the second procedure presented 
above is used, it is possible to refine the abstraction with respect to a transition (qk,Qp), 
where qk — ir ((lk,Dk)) and q p = tt ((l p , D p )), by using the procedure specified by 
algorithm[2] 

At the end of this refinement step, it is possible to reach (l p , D p ) from all states 
of (Ik, D 1 ) and the new discrete transition does not introduce over-approximation. The 
difficulty is obviously to choose the transition that has to be considered for refinement. It 
is then possible to begin with transitions leading to the closest regions to the forbidden 
area [11], or to use the result of the verification and especially the counter-example 
trajectory given by the checking tool, to guide the refinement [ 12]. Another difficulty 
is that a refinement according to one transition is in general not sufficient to conclude 
that (01 is satisfied. So this procedure has to be iterated. If the procedure stops because 
there are no more changes in the discrete system for all transitions, it is eventually 
possible to conclude that if (0 is not satisfied then neither is (0. However there is no 
proof that this convergence will always happen except for specific systems, such as time 
automata Q. 
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Algorithm 2. Discrete abstraction refinement 



Given: < Q,T > a discrete abstraction of AH the hybrid system , 7r its abstraction mapping, 
(qk,q P ) a transition of T such as q k — ty ((l k ,D k )) and q p = 7r ((l p , D p )) 

Result: < Q,T > the discrete abstraction of AH refined with respect to (qk,q p ) and ir its 
abstraction mapping 



Init: Q<= Q, T<=T, S<^7r 



if l h = ; p compute D 1 = D k f] Pred c {l P , D p )|i fc 0and _D 2 = D k - D 1 

if l h / Z p compute D 1 = D k n Pred D (Pred c (l P , A,)),, and D 2 = D - D 1 

ifD 1 /0 and L> 2 /0 then 

delete gfc from Q and g^- = ir ((l ky D k )) from the defintion of ff. Define gjj. = 
-if ((^D 1 )) andg 2 , = 7r ((Z fe ,D 2 )) and add them to Q 
6: Delete (<?*., g p ) from the set of transitions T and add {q kl q P ) to T 
7: for all transitions (g n ,gfc) G T , delete (q n ,q k ) from T and compute whether {q n ,q k ) 

and (gn, ?fc) belong to T 
8: end if 



4.2 Remarks 

This approach that builds a discrete event model of a continuous or hybrid system may 
be found in various propositions B13I14I11I15I16I171 according to the assumptions that 
are made about the system (guards, invariants, continuous dynamics, . . . )• The exact 
specification of the procedures depends on these assumptions and on the choices that 
are made to specify the mapping. In all cases, computation of successor and predecessor 
sets (and especially the continuous ones) is a core problem of the approach. 

5 Reachability for Abstractions 

As it can be seen in the previous sections an important point to build an abstraction is to 
be able to state what are the reachable regions. Two types of answers may be searched 
for. When the problem is to decide whether there is a discrete transition between two 
locations in Sect. [3] or two discrete states in Sect. |4j it is possible to use methods that 
give a yes or no answer, thus characterizing the reachable space without computing it. 
However, it is of course possible to compute the reachable space. When it is necessary 
to refine the abstraction in Sect. [4j it becomes mandatory to get an estimation of the 
predecessor space and then to compute it. It can also be noticed that in both approaches 
these reachability considerations are related to one location and are then mainly con- 
cerned by continuous reachability. 

5.1 Characterizing the Reachable Space 

Families of approaches that characterize the continuous reachable space may be used 
to answer the question whether it is possible to reach a given area from another one. A 
first solution is to find borders that separate the two domains and can't be crossed by 
the continuous trajectories as exemplified on Fig.|7] 

For some specific classes of continuous dynamics it is possible to find generic con- 
straints. For example IfTSII . for linear dynamics without inputs x = A.x, the derivative 
of the linear term p = c T x where c is a left eigenvector of A associated to the the eigen- 
value A, is p — Xp, and hence p — e A *c T Xo for any continuous trajectory. So according 
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Fig. 7. Uncrossable Borders 



Fig. 8. Qualitative invariant 



to the sign of A, the term p is always increasing or decreasing and its minimum or max- 
imum value defines an uncrossable border. As exemplified on Fig. [8] if A is positive the 
region {x|c T x < Minp (c T x)} is unreachable. Such considerations may also be used 
for affine systems with rational eigenvalues 1 19|. 

Another solution that can be used to characterize the reachable space is the concept 
of barrier certificate l20ll . The idea is then to determine a function B that verifies the 
three conditions in (0 as illustrated on Fig. [9| Then the set where this function is equal 
to is an uncrossable border that guarantee that the region P c is unreachable from p> 
With some specific assumptions on the dynamics and the regions it is possible to search 
for this function B by optimization techniques. 



Vx G X, Vu e U B(x) = 



Vx e Pc B(x) > 
Vx e Pa B(x) < 
dB(x) 



dx 



-/(x,u)<0 



(5) 



A last appraoch consists in searching for characteristics of the reachable space in spe- 
cific subspaces, such as the eigensubspaces for linear systems, and then to prove that 
the global set of constraints is inconsistent. This approach will be illustrated in the case 
of linear systems (specified by (x = Ax) and temporal constraints [21 1 but can also be 
used with spacial constraints ll22l . 

Let us consider the problem of reachability of region Pc from region Po an d an 
eigenvalue A of A with an associated eigen subspace with dimension 1 (this case is 
illustrated for 2 dimensions on Fig.fTOll. It is easy to compute, for example with linear 
programming, the upper and lower bounds of the projections of these regions (Po an d 
Pc) on the eigen subspace associated with A, (z , Zq) and (z l c , z c ). In other respects, 
the projection of the trajectory from the point Xo is specified by z(t) = z$e xt where 
Zq is the projection of Xo. It is then possible to compute the minimum and maximum 
time necessary to go from the projection of region Po to the one of Pc. For example 

the maximum time is given by t max .\ = max(y log(^g-), j- log(#)). 

If the maximum time is negative according to this equation, it means that Pc is 
unreachable from Pq as its projection is unreachable in one eigen subspace. If this 
computation is performed with another eigen subspace, it is possible to consider the 
two time intervals. It can then be deduced from the emptiness of their intersection that 
Pc is not reachable from Pq. 
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Fig. 9. Barrier certificate 
5.2 Reachable Space Computation 




Fig. 10. Projections bounds 



When it is necessary to refine a discrete abstraction it is mandatory to compute the pre- 
decessor set of a given region in order to be able to compute its intersection with other 
regions as explained in Sect.[4| The major difficulty is to compute the continuous pre- 
decessor set, or at least an over-approximation, in order to preserve safety conclusions. 

For continuous systems specified by linear differential inclusions, the continuous 
predecessor set can be computed with geometric considerations and polytopes com- 
putations as in the case of the successor set in Sect. [3] (Fig. [3. For other classes of 
continuous dynamics it is possible to consider finite time predecessor sets that can be 
computed by time sampled computations. In general these approaches are devoted to 
successor sets computation but may be adapted to predecessor set for example by con- 
sidering negative time steps. 

Beside approaches based on interval integration of ordinary differential equations 
(23), most approaches are based on the same basic principle that consists in choosing 
a time step S and computing the series of the over-approximations P& of the reachable 
space between times k8 and (fc + 1)5, [8 24 25 26]. The first steps of the method are 
illustrated on Fig. Q~T] when polyhedral regions are chosen. The first step (Fig. [TTIa) 
consists in computing the image, X\, of the initial region, Xq, after time 5. The second 
step (Fig.[TT|b) then consists in searching for a polyhedron Po that includes the whole 
trajectory between the two times. These steps are then iterated to compute the reachable 
space at the next times (Fig.Q~T]c). In order to reduce the approximations, it is generally 
pertinent to use Xi as the basis of the next iteration. However, the second step that 
computes P,_i from Xj_i and Xi is complex and time consuming, so it is also possible 
to adapt this procedure and to compute Pi without computing Xi directly from Pj_i. 
This is especially relevant when this does not induce further approximations as in the 
case of linear dynamics (x = Ax). The evolution from one time step to the other one 
is then known, constant and given by e AS , so, from the computation of Po it is easy to 
iteratively compute the series P; = e A5 Pi-\. 

This approach can also be used for systems specified by x = Ax + u where u stands 
for a bounded uncertainty. At each time step the approximation of the reachable space 
is then given 11241 by Pi = e AS Pi-i © V where V is a region that depends on the un- 
certainty and the dynamics, and © is the Minkowski surrQ of the sets. 



L The Minkowski sum of 2 sets A and B, is denned by A® B = {a + b\ a £ A A b £ B }. 
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Fig. 12. Wrapping effect 



This reachability computation requires computations on regions of the continuous 
state space and then the choice of the representation of these regions. If some works 
consider polynomial regions, (see for example [27 1), classical sets that are considered 
in reachability computation are ellipsoids [28] and various types of polyhedrons such as 
hyperrec tangles (especially with interval computation see e.g. |29|), zonotopes [30 24| 
or general polyhedrons. 

Two main points are to consider in the choice of the region representation. The first 
one is that the set of regions must be closed for the operations that are involved in 
the computation. This is exemplified on Fig. [12] for the hyperrectangle representation 
and the computation of the image by the dynamics. A\, representing the image of A$ 
after one time step, is not a hyperrectangle, so it has to be overapproximated by the 
hyperrectangle in dash line. The result of the second step is the overapproximation of 
the image of this hyperrectangle whereas A^, representing the real image of Aq after 
two time steps, is given by the smaller hyperrectangle. 

A second issue is the complexity of the representation and the possibilities to sim- 
plify it. For example, in the case of polyhedrons the number of constraints that are used 
to describe the region may increase at each step and it may be useful to be able to 
easily compute a tight overapproximation with less constraints in order to simplify the 
computation as examplified on Fig.[T3l 
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Fig. 13. Number of constraints reduction 



Fig. 14. Limiting the number of bits [32] 
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6 Conclusions 

Safety verification of continuous time hybrid systems may be performed by means of 
abstraction, a formal notion that makes it possible to guarantee the relevance of the 
analysis conclusions and to transform a complex problem by mean of a number of 
simpler local problems. One important limitation of the abstraction is that it allows 
checking whether safety properties are satisfied but not always guarantee a solution if 
they are not. The refinement of the model when the analysis is not conclusive is an issue 
that requires some know-how and a crucial point is the formalization of this know-how 
and its integration in tools in order to aid control engineers during safety analysis. 
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Abstract. We have been witnessing numerous world crises and disasters — from 
ecological to military to economic, with global dynamics likely to be increasing 
further. The paper highlights known holistic and gestalt principles mainly used 
for a single brain, extending them to any distributed systems which may need 
high integrity and performance in reaction to unpredictable situations. A higher 
organizational layer is proposed enabling any distributed resources to behave as 
an organism having global "consciousness" and pursuing global goals. This 
"over-operability" layer is established by implanting into key system points the 
same copy of a universal intelligent module, which can communicate with other 
such modules and interpret collectively global mission scenarios presented in a 
special Distributed Scenario Language. The behavioral scenarios can be in- 
jected from any module, and then self-replicate, self-modify, and self-spread 
throughout the system to be managed, tasking individual components, activat- 
ing distributed resources, and establishing runtime infrastructures supporting 
system's integrity. Existing and prospective applications are outlined and dis- 
cussed, confirming the paradigm's suitability for solving world problems. 

Keywords: World crises, Atomism, Holism, Gestalt theory, System integrity, 
Waves, Distributed scenario language, Networked interpretation, Unmanned 
systems, Task level, Behavioral level, Crisis management, Robotized armies, 
Terrorism fight, First computers. 



1 Introduction 

To understand mental state of a handicapped person, problems of economy and ecol- 
ogy, or how to win on a battlefield, we must consider the system as a whole not just 
as a collection and interaction of parts. The situation may complicate dramatically if 
the system is dynamic and open, spreads over large territories, comprises unsafe or 
varying components, and cannot be observed in its entirety from a single point. Nu- 
merous world crises we have been witnessing at the beginning of this century, includ- 
ing the current economic one, may have emerged, first of all, due to our inability of 
seeing and managing complex systems as a whole. 

To withstand the unwanted events and their consequences (ideally: predict and 
prevent them) we need effective worldwide integration of numerous efforts and often 
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dissimilar and scattered resources and systems. Just establishing advanced communi- 
cations between parts of the distributed systems and providing the possibility of shar- 
ing local and global information from different points, often called interoperability, is 
becoming insufficient (even insecure and harmful) for solving urgent problems in 
dynamic environments, in real time and ahead of it. 

We may need the whole distributed system to behave as an integral organism, with 
parts not so interoperating but rather complementing each other and representing 
altogether an integral whole pursuing global goals and having a sort of global aware- 
ness and consciousness. This whole should be essentially more than the sum of its 
parts, with the latter having sense, possibly even existence, in the context of this 
whole, rather than vice versa. 

This paper develops further the over-operability principle researched in [1-4] and 
other works (with the term overoperability coined in [3]), which can establish intelli- 
gent dominant layer over distributed resources and systems and help to solve urgent 
world problems in a parallel, distributed, and dynamic way. 

The rest of this paper compares the dominant atomistic approach in system design, 
implementation and management with holistic and gestalt principles, and describes a 
novel ideology and technology for integral solutions in distributed worlds, which can 
avoid many traditional management routines in solving global problems, with its 
numerous practical applications outlined and discussed. 

2 Atomism, Holism, Gestalt 

We used to exercise predominantly atomistic, parts-to-whole philosophy of the sys- 
tem design, comprehension and implementation, which extends even to the organiza- 
tion of management facilities themselves as a collection of interacting parts, or agents 
(this philosophy actually being the same as a century ago). 

Originally a system or campaign idea and the functionality needed emerge in a 
very general form (in a single human mind or in a close collective of such minds). 
Then this general idea (shown symbolically in Fig. la) is partitioned into individual 
chunks, or "atoms", each detailed and studied further (Fig. lb). This logical partition- 
ing already causes swelling of the problem's complexity (as indicated in the figure). 

The next step is materialization of the defined parts and their distribution in a 
physical or virtual space. To make these parts work or behave together within the 
original idea of Fig. la, we may need a good deal of their communication and 
synchronization, also sophisticated control infrastructures, as depicted in Fig. Ic. 
This overhead may be considerable, outweighing and shadowing the original project 
definition. 

The main problem is that the initial idea (Fig. la) and even its second stage (Fig. 
lb) usually remain in the minds of the creators only, and the real system formaliza- 
tion, description, and implementation are starting from the already partitioned and 
interlinked stage (Fig. Ic), resulting in this huge overhead. 

This parts-to-whole, agents-based approach also dominates in the controversial 
"society of mind" theory [5], which is trying to explain even human thinking from the 
atomistic positions. 
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Fig. 1. Traditional atomistic approach in system design and management, (a) the original sys- 
tem idea; (b) breaking the idea into pieces; (c) system formalization, distribution, and imple- 
mentation. 



Holism [6] has quite an opposite vision of systems: 



• Holism as an idea or philosophical concept is diametrically opposed to atom- 
ism. 

• Where the atomist believes that any whole can be broken down or analyzed 
into its separate parts and the relationships between them, the holist maintains 
that the whole is primary and often greater than the sum of its parts. 

• The atomist divides things up in order to know them better; the holist looks at 
things or systems in aggregate. 

Gestalt theory [7, 8] is based on the holistic principles too: 

• For the gestaltists, "Gestalten" are not the sums of aggregated contents 
erected subjectively upon primarily given pieces. 

• Instead, we are dealing with wholes and whole-processes possessed of inner 
intrinsic laws. 

• Elements are determined as parts by the intrinsic conditions of their wholes 
and are to be understood as parts relative to such wholes." 

Although gestalt psychology and theory was a general approach, most of the work on 
gestalt was done in the area of perception. In our research, we are trying to use the 
holistic and gestalt principles for the organization of distributed systems with highest 
possible integrity and performance [9]. 



3 Waves, Fields, Scenarios 

We describe here a novel organizational philosophy and model, based on the idea of 
spreading interdependent parallel waves (as shown in Fig. 2a), as an alternative to the 
dominant atomistic approach briefed above, staying under the influence of mentioned 
holistic and gestalt principles. 

This allows us for an integral, parallel, and seamless navigation and coverage of vir- 
tual, physical or combined spaces where the solutions need to be found. Atomism 
emerges on the automatic implementation level only, which allows us to get high-level 
formal semantic definitions of systems and global operations in them, while omitting 
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Fig. 2. The waves paradigm, (a) controlled grasping of distributed world with spatial waves; (b) 
self-spreading high-level mission scenarios in distributed networked worlds. 

numerous organizational details (as in Fig. lc) and concentrating on global goals and 
overall performance instead. 

An automatic materialization of this approach is carried out by the network of uni- 
versal intelligent modules (U), embedded into important system points, which collec- 
tively interpret integral mission scenarios expressed in the waves formalism, which 
can start from any U, subsequently covering the distributed system at runtime (see 
Fig. 2b). 

The wavelike scenarios are usually very compact and can be created and modified 
on the fly. They can cooperate or compete with each other in the distributed net- 
worked space as overlapping fields of parallel solutions. 

Spreading waves can create knowledge infrastructures arbitrarily distributed be- 
tween system components (robots, sensors, humans). These, subsequently or simulta- 
neously navigated by same or other waves, can effectively support distributed data- 
bases, command and control, situation awareness, and autonomous decisions. 

This paradigm is much in line with the existing abundant evidence that certain as- 
pects of cognition, morals, needs, object relations, motor skills, and language acquisition 
proceed in developmental stages. These stages appear to be fluid, flowing, overlapping 
waves [10], where: 

• Each stage has a holistic pattern that blends all of its elements into a struc- 
tured whole; 

• These patterns unfold in a relational sequence, with each senior wave tran- 
scending but including its juniors. 

Our approach is also consistent with the ideas of self-actualization and person- 
centered approach [11, 12], where the self is considered as an organized, consistent, 
conceptual gestalt exhibiting active forward thrust — against tension reduction, equi- 
librium, or homeostasis [13]. In our case, instead of a single person we have the 
whole distributed system with high integrity and "active global thrust" behavior. 



4 The Scenario Language 

Distributed Scenario Language, or DSL (similar to is previous versions, WAVE and 
WPL including [2, 4]), reflects the waves model proposed, and allows us to directly 
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express semantics of problems to be solved in distributed worlds, also the needed 
global system behavior in a non-atomistic manner. DSL operates with: 

• Virtual World (VW), which is discrete and consists of nodes and links con- 
necting these nodes. 

• Continuous Physical World (PW), any point in which may be accessed by 
physical coordinates (taking into account certain precision). 

• Virtual-Physical World (VPW), which is an extension of VW where nodes 
additionally associate with certain coordinates in PW. 

It also has the following key features: 

• A DSL scenario develops as a transition between sets of progress points (or 
props) in the form of parallel waves. 

• Starting from a prop, an action may result in one or more props (the resultant 
set of props may include the starting prop too). 

• Each prop has a resulting value (which can be multiple) and a resulting state, 
being one of the four: thru (full success, also allowing us to proceed further), 
done (success with planned termination), fail (regular failure, with local ter- 
mination), and abort (emergency failure, terminating the whole distributed 
process, associated with other props too). 

• Different actions may evolve independently or interdependently from the 
same prop, contributing to (and forming altogether) the resultant set of props. 

• Actions may also spatially succeed each other, with new ones applied in par- 
allel from all the props reached by preceding actions. 

• Elementary operations can directly use local or remote values of props ob- 
tained from other actions (or even from the whole scenarios). 

• Elementary operations can result either in open values that can be directly 
used as operands by other operations in an expression, or by the next opera- 
tions in a sequence. They can also be directly assigned to local or remote 
variables (for the latter case, an access to these variables may invoke scenar- 
ios of any complexity). 

• Any prop can associate with a node in VW or a position in PW, or both — 
when dealing with VPW. 

• Any number of props can be simultaneously linked with the same points of 
the worlds. 

• Staying with world points (virtual, physical, or combined) it is possible to di- 
rectly access and update local data in them. 

• Moving in physical, virtual or combined worlds, with their possible modifica- 
tion or even creation from scratch, are as routine operations as, say, arithme- 
tic or logical operations of traditional programming languages. 

• DSL can also be used as a usual universal programming language (like C, 
Java, or FORTRAN). 

DSL has recursive syntax, being on top level as follows (programs called waves, as 
parallel wavelike movement in distributed spaces dominates the language semantics). 
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wave -} phenomenon I rule ( { wave , }) 

phenomenon -} constant I variable I special 

constant -} information I matter I combined 

variable -} heritable I frontal I environmental I nodal 

rule -} movement I creation I elimination I echoing I fusion I 

verification I assignment I advancing I branching I 

transference I timing I granting 

The basic construct, ra/e, can represent any definition, action or decision, being for 
example: 

• elementary arithmetic, string or logic operation; 

• hop in a physical, virtual, or combined space; 

• hierarchical fusion and return of (remote) data; 

• distributed control, both sequential and parallel; 

• a variety of special contexts for navigation in space, influencing operations and 
decisions; 

• type or sense of a value, or its chosen usage, guiding automatic interpretation. 

There are different types of variables in DSL: 

• Heritable variables - these are starting in a prop and serving all subsequent 
props, which can share them in both read & write operations. 

• Frontal variables - are an individual and exclusive prop's property (not 
shared with other props), being transferred between the consecutive props, 
and replicated if from a single prop a number of props emerge. 

• Environmental variables - are accessing different elements of physical and 
virtual words when navigating them, also a variety of parameters of the inter- 
nal world of DSL interpreter. 

• Nodal variables - allow us to attach an individual temporary property to VW 
and VPW nodes, being accessed and shared by props associated with these 
nodes. 

These variables, especially when used together, allow us to create efficient spatial 
("holographic") algorithms which work in between components of distributed systems 
rather than in them. 

Elementary DSL programming examples are shown in Fig. 3. 

a) assignfResult, add(27, 33,55.6) ) b) movedocationUS.yS) , locationUl,y3 ) ) 
. 27 



Result ■*- 
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d) sequence ( 
c) create (node (Peter) ) hop(Peter), create (+fatherof, Alex)) 
^. ^^ fatherof 
_ _ _► (^ ^ ^ ^ Peter) >-QAlex) 

Fig. 3. Elementary examples in DSL. (a) assignment of a sum of values to a variable; (b) paral- 
lel hop into two physical locations; (c) creating of an isolated node in a virtual space; (d) exten- 
sion of the network with a new link-node pair. 
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5 Composition of Waves 

The language allows for an integral parallel navigation of distributed worlds in a con- 
trolled depth and breadth mode, with any combinations of the two. We will highlight 
here key possibilities of doing this by composition of DSL scenarios, or waves. 

5.1 Single Wave Features 

Single wave (let it be Wl) development features are shown in Fig. 4. Starting from 
a prop, which may be associated with a point in the world, the related scenario 
evolves, grasps, and covers certain region in it, performing any operations needed in 
the distributed space. 

Resultant set of props 

& associated values 
Starting 

prop 




Resultant control 

stete World coverage 

Fig. 4. Single wave features 

The result of this spatial evolution may be multiple, and may lie in a (final) sub- 
region of the region covered, being represented by a set of resultant props (each 
linked to world points) and associated with them values. After termination of the 
wave, its resultant control state (which, in a parallel feedback process, merges termi- 
nation states throughout the region covered) is available in the starting prop, and may 
be taken into account for decisions at higher levels. Also, if requested from higher 
levels, the values associated with the resultant props (which may be remote) can be 
lifted, spatially raked, and returned to the starting prop for further processing. 

5.2 Advancing in Space 

The depth mode development of waves is shown in Fig. 5. For this type of composi- 
tion, each subsequent wave is applied in parallel from all props in space reached by 
the previous wave, with the resultant set of props (and associated values) on the whole 
group being the one of the last applied wave (i.e. W4 in the figure). 
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advance (Wl, W2 , W3,W4) 
Fig. 5. Depth mode composition of waves 

This spatial advancement of waves returns the resultant control state which is 
available at the starting prop, and the values of the resultant set of props can also be 
echoed to the starting prop if requested. Examples of other advancing rules: 

• advance synchronized - the one where any new wave is applied only 
after all invocations of the previous wave have been terminated; 

• repeat - where the same wave is applied repeatedly from all props reached 
by its previous invocation; 

• repeat synchronized - where in the repeated invocation of a wave 
each new invocation starts only after full completion of the previous one. 

5.3 Branching in Space 

The branching breadth mode composition of waves is shown in Fig. 6, where all 
waves in the group are evolving from the same starting prop, and each wave, with its 
own resultant set of props and associated values, contributes to the final result. 



Starting 
prop 

Resultant 

control 

state 




Resultant set 
of props & 
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sequence (W1,W2,W3) 
Fig. 6. Breadth mode composition of waves 
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The merge of results from different waves depends on the branching rule used, with 
their repertoire (besides the sequence in Fig. 7) including: 

if, while, parallel, or, parallel or, and, parallel 
and, cycle, loop, and sling. 

More details on these and other rules can be found, say, from [2, 4]. 

5.4 Combined Branching- Advancing 

Any combination of advancing and branching modes in a distributed space can be 
expressed and implemented in DSL (as shown in Fig. 7). 

Intermediate sets 
of props 



Resultant set 
of props & 
associated 
values 




advance (sequence ( Wl , W2 , W3 ) , W4 ) 
Fig. 7. Breadth-depth composition mode 

These combinations, when embraced by the existing variety of composition 
rules, can provide any imaginable and even so far unimaginable spatial algorithms 
that can solve distributed problems in highly integral and compact ways, without 
explicit descending to the traditional atomistic level, the latter shifted to automatic 
implementation. 



5.5 Operations on Remote Values 

Due to fully recursive organization of DSL, it is possible to program in it arbitrary 
complex expressions directly operating not only on local but also arbitrarily remote 
values, where any programs (scenarios) can happen to be operands of any opera- 
tions (expressed by rules). This gives an enormous expressive power and compact- 
ness to complex spatial scenarios evolving in distributed environments. An example 
of such compact expression of spatial operations on remote values and variables is 
shown in Fig. 8. 
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Fig. 8. Direct operations on remote values 



6 Distributed Interpreter 

The DSL interpreter (as from the previous language versions [1, 2, 4]) has been proto- 
typed on different platforms. Its key features (see Fig. 9) are as follows: 
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Fig. 9. Organization of DSL interpreter 
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It consists of a number of specialized modules working in parallel and han- 
dling and sharing specific data structures supporting persistent virtual worlds 
and temporary hierarchical control mechanisms. 

The whole network of the interpreters can be mobile and open, changing at 
runtime the number of nodes and communication structure between them. 
The heart of the distributed interpreter is its spatial track system, with its 
parts kept in the Track Forest memory of local interpreters (Fig. 9); these be- 
ing logically interlinked with such parts in other interpreter copies, forming 
altogether indivisible space coverage. This enables hierarchical command and 
control and remote data and code access, with high integrity of emerging par- 
allel and distributed solutions, without any centralized resources. 
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• Copies of the interpreter can be concealed, as for acting in hostile systems, al- 
lowing us to impact them overwhelmingly (finding & eliminating unwanted 
infrastructures including). 

The dynamically crated track trees (Fig. 10) spanning the systems in which DSL sce- 
narios evolve (Fig. 10a), are used for supporting spatial variables and echoing and 
merging different types of control states and remote data, being self-optimized in the 
echo processes (Fig. 10b). They also route further waves to the positions in physical, 
virtual or combined spaces reached by the previous waves, uniting them with the 
frontal variables left there by preceding waves. 
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Fig. 10. Distributed track system, (a) forward operations; (b) backward operations. 

7 Programming Examples 

We will show here elementary examples of solution in DSL of some important prob- 
lems on networks and graphs in a fully distributed way, where each node may reside 
in a separate computer. 

7.1 Shortest Paths 



The solution for finding the shortest path between two nodes (a and e) by navigating 
the network with parallel waves is shown in Fig. 11, and by the scenario that follows. 



sequence ( 
(hop (a) ; Ndist 

Fdist += LINK; 
Ndist = Fdist; 



= 0; repeat (hop (all links ) ; 
or (Ndist == nil, Ndist > Fdist) 
Npred = BACK) ) , 



(hop(e); repeat(Fpath 
USER = Fpath) ) 



NAME & Fpath; hop (Npred) 



The result obtained in node a will be as ( a , b , d , e ) 
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Fig. 11. Finding shortest path with waves 



Many important problems of optimization and control may be expressed as finding 
shortest paths in a distributed solution space. 

7.2 Topology Analysis 

DSL allows us to directly analyze and process distributed topologies in a parallel and 
concise way. To find the weakest nodes in a network (like articulation points, see Fig. 
12a) which, when removed, split it into disjoint parts, we only need the following 
program (resulting in node d). 

hop(allnodes) ; IDENTITY = NAME; Mark = 1; 
and( (random(hop (alllinks) ) ; 
repeat (grasp (Mark == nil; Mark = 1); hop (alllinks) )) , 
(hop (alllinks) ; Mark == nil), output (NAME) ) 

Cliques (or fully connected sub-graphs of a graph, as in Figure 12b), on the contrary, 
may be considered as strongest parts of a system. They can be found in parallel by the 
following program, resulting for Figure 12b in: (a, b, c, d), (c, d, e), (d, e, f ). 



Articulation point cf} 1 ? , 




Clique3 




a ) b) 

Fig. 12. Discovering distributed topological features, (a) articulation points; (b) cliques. 
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hop(allnodes) ; Fclique = CONTENT; 

repeat (hop (alllinks) ; no tbe long (CONTENT, Fcligue); 
and(andparallel (hop (anylink, nodes (Fcligue) ) ; done), 
or((BACK > CONT; done), append (Fclique, CONT)))); 
output (Fcligue) 

The above programs work without any central resources, even if every node of the 
network is located in a different computer, with programs starting from any node [2]). 



8 Collective Robotics 

Installing DSL interpreter into mobile robots (ground, aerial, surface, or underwater, 
as in Fig. 13 for the first two) allows us to organize effective group solutions (inch 
any swarming) of complex problems in distributed physical spaces in a concise way, 
shifting traditional management routines to automatic levels. Expressing explicit 
system behaviors at any levels can be easy too. 




RQ4 Global Hawk 



UCJW Grou a aaMalioria 



Fig. 13. Grouping unmanned ground and aerial vehicles 



8.1 Task Level 



Any groups of mobile robots (as in Fig. 13) can be tasked at a highest possible level, 
just telling what they should do together but without detailing how, and what are the 
duties of every unit, which may not be known in advance. An exemplary task here: 

Go to physical locations of the disaster zone with coordinates (50.433, 30.633), 
(50.417, 30.490), and (50.467, 30.517). Evaluate damage in each location, find and 
transmit the maximum destruction value, together with exact coordinates of the corre- 
sponding location, to a management center. 

The DSL program will be as follows: 

transmit (max ( 

move( (50.433,3 0.633) , (50 . 417 , 30 . 490) , (50 . 467 , 30 . 517 ) ) ,- 
append (evaluate (destruction) , WHERE) ) ) 

Details of automatic implementation of this scenario by different numbers of mobile 
robots are discussed elsewhere [14]. 
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8.2 Behavior Level 

After embedding DSL interpreters into robotic vehicles, we can also provide any 
needed detailed collective behavior of them (at a lower than top task level, as before)- 
from loose swarms to a strictly controlled integral unit obeying external orders. Any 
mixture of different behaviors within the same scenario can be easily programmed 
too. 

The following DSL scenario combines loose, random swarm movement in a dis- 
tributed space with periodic finding/updating topologically central unit, and setting 
runtime hierarchical infrastructure between the units. The latter controls observation 
of distributed territory, collecting potential targets, distributing them between the 
vehicles, and then impacting potential targets by them individually. More on the 
implementation of this scenario can be found in [14, 15]. 

(hop (allnodes) ; Range = 500; 
Limits = (dx(0,8), dy(-2,5)); 
repeat(Shift = random (Limits) ; 
if (empty (hop (Shift, Range), move (Shift) ))) , 
(repeat (hop (Faver = average (hop (allnodes) ,- WHERE) ,- 
min (hop (allnodes) ; distance (Aver , WHERE)& ADDR):2)); 
stay (hop (allnodes) ; remove (alllinks) ; 
Frange = 20; repeat (linkup (+infra, first, Frange) ) ; 
orpar (loop (nonempty (Fseen = 
repeat (free (detect (targets) ) , hoplinks (+inf ra) ) ; 
repeat (free (select_move_shoot (Fseen) , 
hoplinks (+infra) )) , sleep(360))) 



9 Other Applications 

A variety of other potential applications of the proposed paradigm have been investi- 
gated and published [14-22]. We are mentioning here just two of them, currently in 
active research and assessment. 

9.1 Robotized Armies 

Distributed robotized systems are of rapidly growing importance in defense [23], 
where robotic swarming on asymmetric battlefields is becoming a major dimension of 
the new military doctrine for 21 st century. But, as admitted in [23], swarming, along 
with its simple rules of individual behavior and fully distributed nature, agility, and 
ubiquity, may also result in unpredictability of behavior for both sides of the conflict. 

The approach briefed in this paper, also investigated in previous publications on 
this paradigm, is very much in line with these modern trends. Moreover, we are offer- 
ing a unified solution that can harness loosely coupled swarms, always guaranteeing 
their global-goal-driven behavior, where the watershed between loose swarming and 
strict hierarchical control may be situation dependent and changing over time. 

These new doctrine trends will inevitably influence the role and sense of commu- 
nications on battlefields, as with the planned drastic reduction of centralized C2 much 
more emphasis will be paid to intelligent tactical communications, where the scenario 
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mobility in networked systems, offered by the approach proposed, may constitute an 
effective solution, with the key points [17] as follows: 

• Dramatic shift of global organization to intelligent tactical communications; 

• Self-spreading and self-recovering mission scenarios and emergent command 
and control; 

• Embedding intelligent protocol module into existing communication equip- 
ment; 

• Situation-dependent watershed between global control and local communica- 
tions. 

In relation to the mentioned above, different (including new) types of command and 
control strategies for distributed robotized systems have been investigated in DSL [20]. 

9.2 Terrorism and Piracy Fight 

It is no secret that the mightiest world armies with most powerful weapons and 
wonderful classical organizations are often losing to poorly armed terrorists and in- 
surgents, also are powerless before the sea piracy, who are using very flexible, asym- 
metric tactics. A fresh piracy map is shown in Fig. 14, with piracy cases only for 2009 
throughout the world, and possible information sources they are using, which form a 
sophisticated distributed network. The ideology and technology discussed in this 
paper can dynamically organize the whole world to withstand these activities, offering 
runtime (and ahead of it) spatial solutions, with high integrity and goal orientation — 
from global networks search to organizing smart unmanned swarms providing asym- 
metric responses to the asymmetric attacks. The approach can guarantee survivability 
of missions and fulfilling global and local objectives under any circumstances. 
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Fig. 14. Global piracy fight 
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There are a number of initiatives to utilize unmanned aircraft swarms to counter the 
threat of sea pirates, who also attack with swarming techniques [24]. The swarm will 
recon a wide region ahead of the chopper and converge to assault any emerging 
targets. The project seeks to make the swarm fully autonomous so that it can identify 
targets and choose which ones to attack. This would enable the vehicles and devices 
to act on their own, in an autonomous manner, based upon data sent from their own 
swarm or other swarms. An operational version of the network could help direct 
swarms of unmanned boats, helicopters or jet-powered aircraft. 

All these initiatives can be effectively programmed and managed in DSL. 

10 Comparison with the First Computers 

The approach, offering an advanced system organization, can be compared with the 
invention of the first world computers [25] and first high-level programming lan- 
guages [26]. In our case, this computer may not only operate with data stored in a 
localized memory, but can cover, grasp, and manage any distributed system, the 
whole world including, and can work not only with information but with physical 
matter or objects too. 

If compared with the Turing computational model, instead of the head moving 
through tape in performing calculations, we have a recursive formula that unwraps, 
replicates, covers and matches the distributed world in parallel, scanning it forth and 
back, bringing operations and data directly to the points of their consumption, auto- 
matically setting distributed command and control infrastructures, and organizing 
local and global behaviors needed. The term "computer" first referred to the people 
who did scientific calculations by hand [27]. In the end, they were rewarded by a new 
electronic machine that took the place and the name of those who were, once, the 
computers. 

We can draw the following symbolic parallel with this. Despite the overwhelming 
automation of human activity (in both information and matter processing) the world 
as a whole may still be considered as a human machine, as main decisions and global 
management still remain the human prerogative. With the approach offered, we can 
effectively automate this top-level human supervision, actually converting the whole 
world into a universal programmable machine spatially executing global scenarios in 
DSL or a similar language. Despite certain science fiction flavor of this comparison, 
we can find numerous applications for such a global approach, some mentioned 
above, where top level decision automation could withstand unwanted events and 
save lives, and where timely human reaction may not be possible, even in principle. 

11 Conclusions 

We have developed and tested a novel system approach, which can describe what the 
system should do and how to behave on a higher level, while delegating traditional 
management details (like partitioning into components, their distribution, interaction 
and synchronization) to the effective automatic layer. 
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A DSL scenario is not a usual program — it is rather a recursive active spatial pat- 
tern dynamically matching structures of distributed worlds. It has a hierarchical or- 
ganization, which is grasping, by means of spreading parallel waves, the whole of the 
system to be comprehended and impacted. 

The DSL scenarios can also create, in a parallel and fully distributed way, active 
distributed worlds, which become persistent and operate independently. They may 
spatially intervene into operation of these and other worlds and systems, changing 
their structures and behaviors in the way required, also self-recover from indiscrimi- 
nate failures and damages, as well as repair and recover the systems managed. 

Prospective applications of this work can also be linked with economy, ecology 
and weather prediction — by using the whole networked world as a spatial supercom- 
puter, self-optimizing its performance. Also, for terrorism and piracy fight, where the 
powerful parallel ability of analyzing distributed systems and finding strong and weak 
patterns in them (as shown in Section 7.2), as well as any structures, may be the key 
to global solutions. 

Crises may emerge anywhere and anytime like, say, birds or swine flu or the cur- 
rent global economic disaster. We must be ready to react on them quickly and asym- 
metrically, withstanding and eradicating them — in a "pandemic" way too, highly 
organized and intelligent, however. 

We already have technical capabilities for this, as for example, the number of mo- 
bile phone owners in the world is approaching 3bn, and installing DSL interpreter in at 
least a fraction of them, can allow us to organize collective runtime (and ahead of it) 
response to any world events. 

Acknowledgements. Special thanks to organizers of the international ICINCO con- 
ferences for their regular support of these ideas, and particularly to Joaquim Filipe, 
Marina Carvalho and Vitor Pedrosa. 
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Abstract. In this paper an improved Mean-shift tracking algorithm based on 
adaptive multiple feature fusion is presented. A two-class variance ratio is em- 
ployed to measure the discriminate between object and background. The multi- 
ple features are Fused by linear weighting to realise Mean-shift tracking using 
the discrimination. Furthermore, an adaptive model updating mechanism based 
on the likelihood of the features between successive frames is addressed to alle- 
viate the mode drifts. Based on biology vision theory ,colour, edge and texture cue 
are employed to implement the scheme. Experiments on several video sequences 
show the effectiveness of the proposed method. 



1 Introduction 

Visual tracking is a common task in computer vision and play key roles in many sci- 
entific and engineering fields. Various applications ranging from video surveillance, 
human computer interaction, traffic monitoring to video analysis and understanding, all 
require the ability to track objects in a complex scene. Many powerful algorithms for 
target tracking have yielded two decades of vision research. 

Mean shift algorithm is a nonparametric kernel density estimator based on the colour 
kernel density estimation, which has recently gained more attention due to its low com- 
putational complexity and robustness. Fukunaga and Hostetler first proposed the mean 
shift algorithm [ 1 1 for clustering data in 1975, and Cheng [2 1 introduced it to the image- 
processing community a decade ago. Mean shift algorithm to image segmentation and 
tracking [7 8 9111112] is proposed by Commanicui et al. Later, many variants of the 
mean shift algorithms were proposed for different applications I10I11I12I13I14I151 . 

Frame difference and adaptive background subtraction combined with simple data 
association techniques can effectively track in real-time for stationary cameras target 
tracking [3 4 5 1. Optical flow methods using the pattern of apparent motion of objects, 
surfaces and edges in a visual scene caused by the relative motion between the cam- 
era and scene. These methods can achieve the target tracking in the stationary cameras 
scene and the mobile cameras scene [16]. Modern appearance-based methods using 
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the likelihood between the tracked target appearance describe model and the reference 
target appearance describe model can achieve the target tracking without prior knowl- 
edge of scene structure or camera motion. Modern appearance-based methods include 
the use of flexible template models [2 7| and kernel-based methods that track nonrigid 
objects used colour histograms 11819110 ]. Particle filter and Kalman filter are using to 
achieve more robust tracking of manoeuvring objects by introducing statistical models 
of object and camera motion 19I11I12I13I1 . 

The major difficulty in target tracking based on computer vision is the variation of 
the target appearance and its background. By using a stationary camera, the background 
in a long image sequence is dynamic. However, the performance of a tracker can be im- 
proved by using an adaptive tracking scheme and multiple features. The basic ideal is 
online adaptive selection of appropriate features for tracking. Recently, several adaptive 
tracking algorithms lll4ll5ll6ll7ll8l were proposed. Collins et al. [ 14 1 proposed to on- 
line select discriminative tracking features from linear combination of RGB values. The 
two-class variance ratio is used to rank each feature by how well it separates the sample 
distributions of object and background. Top N features that have the greatest discrimina- 
tion are selected to embed in a mean-shift tracking system. This approach,howeveronly 
considers the RGB colour features. Actually, this approach is one feature-based track- 
ing. Furthermore, it lacks an effective model update method to copy with the model 
drifts. Liang et al. [ 17 1 extended the work of Collins et al. by introducing adaptive fea- 
ture selection and scale adaptation. A new feature selection method based on Bayes 
error rate is proposed. But how to deal with the model drifts is not addressed in this 
paper. He et al. [ 16] used a clustering method to segment the object tracking according 
to different colours, and generate a Gaussian model for each segment respectively to 
extract the colour feature. Then an appropriate model was selected by judging the dis- 
crimination of the features. The Gaussian model however, not always fit each segment 
in practice. Recently, Wang and Yagi |15| extended the standard mean-shift tracking 
algorithm to an adaptive tracker by selecting reliable features from RGB, HSV, nor- 
malised RG colour cues and shape-texture cues, according to their descriptive ability. 
But only two best discriminate features are used to represent the target. It dose not use 
fully all the features information it has computed and has a high time complexity. 

A key issue addressed in this work is an online, adaptive multiple-feature fusion and 
template-update mechanism for target tracking. Based on the theory of biologically vi- 
sual recognition system, the main visual information comes from the colour feature, 
edge feature and the texture feature [19 20 21]. In this paper, RGB colour features, 
Prewitt edge feature and local binary pattern (LBP) texture feature are employed to 
implement the scheme. Target tracking is considered as a local discrimination prob- 
lem with two classes: foreground and background. Many works have point the features 
that best discriminate between object and background are also best for tracking per- 
formance [14 19 20|. In this paper the tracked target is represented by a fused feature. 
According to the discriminate between object and background measured by two-class 
variance ratio, the multiple features are combined by linear weighting to realise kernel- 
based tracking. As model drafts, better performance could be achieved by using a novel 
up-dating strategy that takes into account the similarity between the initial and cur- 
rent appearance of the target. Each feature's similarity is computed. The high similarity 
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features are given a big weight and the low similarity features are given a small weight. 
A good feature for tracking is a steady feature across the consecutive frames. The target 
update model is updated by re-weighting the multiple features based on the similar- 
ity between the initial and current appearance of the target. The proposed approach is 
shown as Figure 1 . 
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Fig. 1. The Flow chart of the purposed approach 

The paper is organised as follows. Section 2 presents a brief introduction the feature 
subset selection. And Section 3 presents our proposed approach for target tracking. 
Computer simulation and results compared with related work are presented in Section 
4. Concluding remarks are given in Section 5. 

2 Feature Subset Selection 



It is important to decide what kinds of features are used before constructing the feature 
fusion mechanism. In ifPfll . the set of candidate features is composed of linear combina- 
tion of RGB pixel. In B15I . colour cue and shape-texture cues are employed to describe 
the model of the target. In this paper based on the theory of biologically visual recog- 
nition system [ 19 20 21 1, RGB colour features, Prewitt edge feature and local binary 
pattern (LBP) texture feature are employed to implement the scheme. 



2.1 RGB Colour Feature 

Colour information is an important visual feature. That is robust to the target rotary, 
non-rigid transformation and target shelter, widely used in the appearance model-based 
visual application. In this paper, colour distributions are represented by colour his- 
tograms, and RGB colour space as a very common colour space is used in this pa- 
per. The R, G, and B channels are quantised into 256 bins, respectively. The colour 
histogram, calculated using Epanechnikov kernel, is applied [8 9|. 



2.2 Prewitt Edge Feature 

The edge information is the most fundamental characteristic of images. It is also 
included useful information for target tracking. There are many methods for edge de- 
tection, but most of them can be grouped into two categories: search-based and zero- 
crossing based. The search-based methods detect edges by first computing a measure 
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of edge strength, usually a first-order derivative expression, such as the gradient magni- 
tude; and then searching for local directional maxima of the gradient magnitude using 
a computed estimate of the local orientation of the edge, usually the gradient direction 
[22 23 1 . The zero-crossing based methods search for zero crossings in a second-order 
derivative expression computed from the image, in order to find edges, usually the zero- 
crossings of the Laplacian or the zero-crossings of a non-linear differential expression 

m. 

In this paper, Prewitt operator is employed to get the edge feature. For its low compu- 
tational complexity and high performance. Prewitt operator has two convolution kernels 
as shown in Figure 2. Images of each point are used for the convolution kernel, the first 
kernel usually corresponding to the largest vertical edge, and the second corresponding 
to the largest horizontal edge. The maximum values of each point convoluted with the 
two kernels are accepted. Convolution is admitted as output value; results of operations 
are edge images. The Prewitt operator can be defined as: 



S p — <Jd x + d y , 



(1) 



d x =[f(x - l,y - 1) + fix, y-l) + f{x + 1, y - 1)]- 
[f(x - 1, y + 1) + f(x, y+l) + f{x + 1, y + 1)], 



(2) 



=[/(* + 1,7/- 1) + f(x + l,y) + f{x + 1, y + 1)]- 
[fix - l,y - 1) + fix -l,y) + fix - 1,1, + 1)]. 



(3) 



The histogram is used to represent the edge feature. Prewitt edge is also quantised into 
256 bins Epanechnikov kernel like colour feature. 
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Fig. 2. The two convolution kernel of Prewitt 



2.3 LBP Texture Feature 



Local Binary Patterns (LBP) is basically a fine-scale descriptor that captures small tex- 
ture details. It is also very resistant to lighting changes. LBP is a good choice for coding 
fine details of facial appearance and texture [24 25 26]. The Local Binary Patterns op- 
erator is introduced as a means of summarising local gray-level structure by Ojiala in 
1996 [26). The operator takes a local neighbourhood around each pixel, thresholds the 
pixels of the neighbourhood at the value of the central pixel, and uses the resulting 
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binary-valued image patch as a local image descriptor. It was originally defined for 
3*3 neighbourhoods, given 8 bit codes based on the 8 pixels around the central one. 
Formally, the LBP operator takes the form 



LBP{x k ,y k ) = Y, TS ( l 



lk) 



(4) 



S(u) 



1. 



> th, 



0, u < th, 



(5) 



where in this case n runs over the 8 neighbours of the central pixel k, ik and i n are 
the gray-level values at k and n, and S(u) is 1 if X > th and otherwise. The LBP 
encoding process is illustrated in Figure 3. 
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Fig. 3. Illustration of the basic LBP operator 

In methods that turn LBPs into histograms, the number of bins can be reduced sig- 
nificantly by assigning all non-uniform patterns to a single bin, often without losing too 
much information. In this paper, it is quantised into 256 bins with Epanechnikov kernel. 

3 Online, Adaptive Features Fusion Method for Tracking 

There are two main components in this approach: the online adaptive features fusion 
based on discrimination criterion function, and the kernel-based tracking, which is used 
to track targets, based on the fused feature. 



3.1 Features Fusion Method 

In this paper, the target is represented by a rectangular set of pixels covering the tar- 
get, while the background is represented by a larger surrounding ring of pixels. Given 
a feature /, let Hf g (i) be a histogram of target and Hf, g (i) be a histogram for the 
background. The empirical discrete probability distribution p(i) for the object and q(i) 
for the background, can be calculated as p(i) — Hf g (i)/nf g andg(i) = Hb g {i) / rib g 
, where nf g is the pixel number of the target region and nt, g the pixel number of the 
background. The weight histograms represent the features only. It does not reflect the 
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descriptive ability of the features directly. A log-likelihood ratio image is employed to 
solve this problem 0141151 . The likelihood ratio nonlinear log likelihood ratio maps fea- 
ture values associated with the target to positive values and those associated with the 
background to negative values. The likelihood ratio of a feature is given by 

rr\ I 1 ■ n i ^ ™az(p(i),e) 

L(i) = max( — 1, rmnll, tog( r~r\ — \")))> (°) 

max(q{i),e) 

where e is a very small number( set in 0.001 in this work), that prevents dividing by 
zero or taking the log of zero. Likelihood ratio images are the foundation for evaluating 
the discriminative ability of the features in the candidate features set. Figure 4 shows 
the likelihood ratio images of different features. 

In the practice, the whole weighted images weighted by log likelihood are not needed 
to be calculated for the computational complexity. The corresponding variance is em- 
ployed to measure the separately between target and background classes. Using the 
method in 11141151 . based on the equality var(x) — E[x 2 ] — (E[x]) 2 , the variance of 
the log likelihood is computed as 

var(L:p) = E[L(i) 2 ]-(E[L(i)]) 2 . (7) 

The discriminative ability of each feature is calculated by the variance ratio. The hy- 
pothesis in this paper is that the features that best discriminate between target and back- 
ground are also best for tracking the target. So, as the target's features describe model, 
a highest weight is given to the best discriminate feature, and the less discriminate 
feature has a smaller contribution. Based on the discrimination criterion function , the 
target features describe model can be calculated as 

n 

Pt(i) =y~]^kPk(i), (8) 

fc=i 
_ var k (L:p) 
2^ k=1 var k {L :p) 

where Pk(i) is the feature K's probability distribution model, A^is the weight and 
Sfc=i ^fc = 1 • Figure 5 shows the fusion of the five features image. 



3.2 Kernel-Based Tracking 

Mean shift is a nonparametric kernel density estimator, which, based on the colour ker- 
nel density estimation, has recently gained more attention due to its low computational 
complexity and robustness to appearance change, however, the basic mean shift track- 
ing algorithm assumes that the target representation is discriminative enough against 
the background. This assumption is not always true, especially when tracking is carried 
out in a dynamic background [9 10] . An online, adaptive features fusion mechanism is 
embedded in the kernel-based mean shift algorithm for effective tracking. Due to the 
continuous nature of video, the distribution of target and background features in the 
current frame should remain similar to the previous frame and the fused feature model 
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Fig. 4. Different feature image after likelihood ratio process, (a) The input frame, (b) feature R 
and likelihood ratio, (c) feature G and likelihood ratio, (d) feature B and likelihood ratio, (e) LBP 
texture feature and likelihood ratio, (f) Prewitt edge feature and likelihood ratio. 



should still be valid. The initial position of the target is given by yO which is determined 
in the previous frame. The target model is P — Pt, t —i , Y^tLi Pt = 1 > an d the candi- 
date target model is P(yo) — Pt(yo)t=i m'^2t^iPt ~ 1, where p t is the fused feature 
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Fig. 5. The image after fusion 

model. The Epanechnikou profile [8, 9] is employed in this paper. The target's shift 
vector form yo in the current frame is computed as 

V EL\^(II^II 2 ) ' 

where g(x) — —k'(x) ,k(x) is Epanechnikou profile, h is bandwidth and u>ican com- 
pute as 

And the tracker assigns a new position of the target by using 

yi = ^{yv + yi)- (12) 

If 1 1 j/o — 2/i II > the iteration computation stops and j/i is taken as the position of the target 
in the current fame. Otherwise let j/o = 2/i» then using Eq. (10) get the shift vector and 
do position assignment using Eq. (12). From Eq. (8) and Eq. (10), the pixels' weight is 
assigned by two parts. One is the kernel profile, which gives high weight to the pixel 
nearly to the centre. The other one is the discriminative ability of each feature. Higher 
weight is given to the higher discriminative ability feature. 

3.3 Template Update Mechanism 

It is necessary to update the target model, because the variation of the target appearance 
and its background. When the target appearance or the background changes, the fixed 
target model can not accurate describe the target, so it can not obtain the right position 
of target. But using an inaccurately tracking result to update the target model may lead 
to the wrong update of the target model. And with the error accumulate, it finally results 
in track failure. 

In order to alleviate the mode drifts, an adaptive model update mechanism based 
on the likelihood of the features between successive frames is proposed in this paper. 
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During the initialisation stage, the target is obtained by a hand-draw rectangle and the 
target model is computed by the fusion method introduced in the previous subsection. 
The fused target model is used for tracking in the next frame and is also kept to use in 
subsequent model updates. Fellowing the method in 1 15 1, the updated target model M 
can be computed as 

M = (1 - L ic )Mi + L ic M c , (13) 

L ic = Y, yjMiM c , (14) 

u=l 

where Li C is the likelihood between the initial model and current model measured by 
Bhattacharyya coefficient II8I9I : Mi is the initial model; M c is the current target model 
computed as 

M c = (1 - L pc )M p + L pc M a , (15) 

where M p is the previous target model, M a is the current target fused-feature intro- 
duced in section 3, and L pc is computed the likelihood between the M a and M p . The 
proposed updating method considers temporal coherence by weighing the initial target 
model, previous target model and current candidate. It can be more robust for the target 
appearance and the background change. 



4 Experiments 

To illustrate the benefits of the proposed approach, experiments on various test video 
sequences using the proposed approach and other algorithms are conducted. The ex- 
periment was done using Pentium core 1.8G, Win XP, Matlab 7.0. The Epanechnikov 
profile was used for histogram computations. The RGB colour feature, LBP texture fea- 
ture and Prewitt edge feature were taken as feature space and it was quantised into 256 
bins. The public dataset with ground truth [27 1 is used to test the proposal. The track- 
ing results are compared with the basic mean-shift [8], fore/background ratio, variance 
ratio, peak difference [9], and multiple feature [15| tracker. The initialisation of the 
tracking is given by a hand-draw rectangle. The same initialisations are given to all the 
trackers. In this experiment, sequences EgTestOl, EgTest02, EgTest03, EgTest04 and 
EgTest05 in the database are chosen. This challenging tracking sequences is made by 
various factors, such as different viewpoints, illumination changes, reflectance varia- 
tions of the targets, similar objects nearby, and partial occlusions. The Tracking results 
of sequences EgTestOl, EgTest02, EgTest03, EgTest04 and EgTest05 as Fig. 6, Fig. 7, 
Fig. 8, Fig. 9 and Fig. 10 respectively. While the red rectangle is the tracking window. 

The tracking success rate is the most important criterion for target tracking, which 
is the number of successful tracked frames divided by the total number of frames. The 
bounding box that overlaps the ground truth can be considered as a successful track. 
To demonstrate the accuracy of tracking, the average overlap between bounding boxes 
(Avg overlap BB) and average overlap between bitmaps within the overlapping bound- 
ing box area (Avg overlap BM) are employed. Avg overlap BB is the percentage of the 
overlap between the tracking bounding box and the bounding box identified by ground 
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Fig. 6. The tracking result of sequences EgTestOl using proposed algorithm, (a) The initial frame, 
(b) The 50th frame tracking result, (c) The 100th frame tracking result, (d) The 150th frame 
tracking result, (e) The 200th frame tracking result, (f) The 250th frame tracking result, (g) The 
300th frame tracking result, (h) The 350th frame tracking result. 




Fig. 7. The tracking result of sequences EgTest02 using proposed algorithm, (a) The initial frame, 
(b) The 50th frame tracking result, (c) The 100th frame tracking result, (d) The 200th frame 
tracking result, (e) The 300th frame tracking result, (f) The 400th frame tracking result, (g) The 
500th frame tracking result, (h) The 600th frame tracking result. 




Fig. 8. The tracking result of sequences EgTest03 using proposed algorithm, (a) The initial frame, 
(b) The 50th frame tracking result, (c) The 100th frame tracking result, (d) The 200th frame 
tracking result, (e) The 300th frame tracking result, (f) The 400th frame tracking result, (g) The 
500th frame tracking result, (h) The 600th frame tracking result. 



truth files. Avg overlap BM is computed in the area of the intersection between the user 
bounding box and the ground truth bounding box. The comparison results are shown in 
Table 1. 
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Fig. 9. The tracking result of sequences EgTest04 using proposed algorithm, (a) The initial frame, 
(b) The 50th frame tracking result, (c) The 100th frame tracking result, (d) The 200th frame 
tracking result, (e) The 300th frame tracking result, (f) The 400th frame tracking result, (g) The 
500th frame tracking result, (h) The 600th frame tracking result. 
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Fig. 10. The tracking result of sequences EgTest05 using proposed algorithm, (a) The initial 
frame, (b) The 50th frame tracking result, (c) The 100th frame tracking result, (d) The 150th 
frame tracking result, (e) The 200th frame tracking result, (f) The 250th frame tracking result, (g) 
The 300th frame tracking result, (h) The 350th frame tracking result. 



From the comparison results that show the successful tracking ratio, the proposed 
tracker gives the best results in five of the test sequences. The basic mean-shift tracker 
dose not have a good performance in EgTestOl, EgTest02 and EgTest05, because the 
basic mean-shift tracker dose not use the multi-features information and lacks adaptive 
strategy. 

Although an adaptive strategy is employed in Collins's approach, it dose not have 
good performance in EgTestOl, EgTest02 and EgTest03. The peak difference algo- 
rithm has a better performance in the first sequences. The others sequences, however, do 
not demonstrate a good performance either. The multi-features methods that Integrate 
Colour and Shape-Texture features have a higher performance in all the sequences. Al- 
though the proposed approach has the best performance than the other trackers. But in 
EgTest03, EgTest04 and EgTest05 the successful tracking ratio is only 25.30%, 12.03% 
and 24.3 1 %. The failed tracking examples are show as Figure 1 1 .The main reason lead- 
ing to tracking failure includes the similar feature distribution nearby as (a) in Fig. 11, 
the lower discrimination between foreground and background as (b) in Fig. 1 1 and long 
time occlusions as (c) in Fig.l 1. 
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Table 1. Tracking performance of different Algorithm. (a)EgTestOl. (b)EgTest02. (c)EgTest03. 
(d)EgTest04. (e)EgTest05. 



Criterion 


MeanShift 


FgBgRatio 


VarianceRatio 


PeakDiff 


Multi-feature 


The Proposed 


Successful ratio 


17.58% 


100% 


29.12% 


100% 


100% 


100% 


Avg overlap BB 


65.50% 


62.87% 


76.87% 


61.76% 


61.62% 


74.28% 


Avg overlap BM 


66.26% 


49.15% 


61.30% 


57.76% 


68.38% 


76.56% 



(a) 



Criterion 


MeanShift 


FgBgRatio 


VarianceRatio 


PeakDiff 


Multi-feature 


The Proposed 


Sucessful ratio 


39.23% 


39.23% 


27.69% 


30.77% 


100% 


100% 


Avg overlap BB 


91.09% 


89.13% 


85.19% 


90.54% 


93.32% 


94.21% 


Avg overlap BM 


74.69% 


66.98% 


73.32% 


65.91% 


72.70% 


73.53% 



(b) 



Criterion 


MeanShift 


FgBgRatio 


VarianceRatio 


PeakDiff 


Multi-feature 


The Proposed 


Sucessful ratio 


20.62% 


17.90% 


12.06% 


12.06% 


20.23% 


25.30% 


Avg overlap BB 


86.96% 


87.01% 


93.74% 


92.27% 


88.66% 


90.11% 


Avg overlap BM 


66.65% 


54.04% 


70.79% 


67.20% 


69.37% 


71.23% 



(c) 



Criterion 


MeanShift 


FgBgRatio 


VarianceRatio 


PeakDiff 


Multi-feature 


The Proposed 


Sucessful ratio 


9.84% 


8.74% 


9.84% 


3.83% 


9.84% 


12.03% 


Avg overlap BB 


66.78% 


67.92% 


66.03% 


63.60% 


69.52% 


71.00% 


Avg overlap BM 


59.70% 


52.75% 


66.74% 


66.42% 


56.34% 


68.43% 



(d) 



Criterion 


MeanShift 


FgBgRatio 


VarianceRatio 


PeakDiff 


Multi-feature 


The Proposed 


Sucessful ratio 


13.64% 


13.64% 


13.64% 


13.64% 


21.22% 


24.31% 


Avg overlap BB 


94.58% 


88.75% 


86.46% 


86.98% 


72.65% 


75.41% 


Avg overlap BM 


84.02% 


71.12% 


85.12% 


69.90% 


64.45% 


70.03% 



(e) 




Fig. 11. The failure tracking examples of the proposed approach, (a) and (b) show the failure 
frame in EgTest03, (c) show the failure result in EgTest04. 
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For accuracy of tracking, the proposed tracking algorithm is not the best in some of 
the sequences. There is not obvious correlation between the tracking accuracy criterion 
and the tracking successful ratio. The proposed approach does not have the highest ac- 
curacy, because in most frame-sequences,the background and the target are not always 
separated accurately . 

5 Conclusions 

An online, adaptive multiple features fusion and template update mechanism for kernel- 
based target tracking is addressed in this paper. Based on the theory of biologically 
visual recognition system, RGB colour features, Prewitt edge feature and local binary 
pattern (LBP) texture feature are employed to implement the scheme. Experiment re- 
sults show that the proposed approach is effective for target tracking. The comparison 
with other algorithms proved that the proposed approach has the best performance for 
target tracking. 

Acknowledgements. This work is supported by China Scholarship Council. This work 
also supported by Chongqing University, "211 Project" three creative talents training 
plan construction projects (S-09108). 

References 

1. Nir, T., Alfered, M.: Over-Parameterized Variational Optical Flow. International Journal of 
Computer Vision 76, 205-216 (2008) 

2. Junqiu, W., Yasushi, Y: Integrating Color and Shape-Texture Features for Adaptive Real- 
Time Object Tracking. IEEE Transactions on Image Processing 17, 235-240 (2008) 

3. Collins, R., Lipton, A., Fujiyoshi, H., Kanade, T: Algorithms for Cooperative Multi-Sensor 
Surveillance. Proceedings of the IEEE 89, 1456-1477 (2001) 

4. Bar, Y, Fortmann, T: Tracking and Data Association. Academic Press, London (1988) 

5. Stauffer, C, Grimson, W.E.L.: Learning Patterns of Activity Using Real-Time Tracking. 
IEEE Transactions on Pattern Analysis and Machine Intelligence 17, 814-820 (1995) 

6. Barron, J.L., Fleet, D.J.: Performance of optical flow techniques. International Journal of 
Computer Vision 12, 43-77 (1994) 

7. Mattews, I., Iashikawa, T, Baker, S.: The Template Update Problem. IEEE Transactions on 
Pattern Analysis and Machine Intelligence 26, 810-815 (2004) 

8. Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE 
Transactions on Pattern Analysis and Machine Intelligence 24, 603-619 (2002) 

9. Comaniciu, D., Ramesh, V, Meer, P.: Kernel-based object tracking. IEEE Transactions on 
Pattern Analysis and Machine Intelligence 25, 564-577 (2003) 

10. Li, Z., Tang, Q., Sanq, N.: Improved meanshift algorithm for occlusion pedestrian tracking. 
Electronics Letters 44, 622-623 (2008) 

11. Pan, J., Hu, B., Zhang, J.: Robust and Accurate Object Tracking Under Various Types of 
Occlusions. IEEE Transactions on Circuits and Systems for Video Technology 12, 223-236 
(2008) 

12. Chang, W.-Y, Chen, C.-S., Jian, Y.-D.: Visual Tracking in High Dimensional State Space 
by Appearance-Guided Particle Filtering. IEEE Transactions on Image Processing 17, 1 154- 
1167(2008) 



62 H. Yin et al. 

13. Maggio, E., Smerladi, F„ Cavallaro, A.: Adaptive Multifeature Tracking in a particle Filtering 
Framework. IEEE Transactions on Circuits and Systems for Video Technology 12, 1348- 
1359 (2007) 

14. Collins, T, Liu, Y., Leordeanu, M.: Online Selection of Discriminative Tracking Features. 
IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 1631-1643 (2005) 

15. Wang, J., Yagi, Y: Integrating Color and Shape-Texture Features for Adaptive Real-Time 
Object Tracking. IEEE Transactions on Image Processing 17, 235-240 (2008) 

16. He, W., Zhao, X., Zhang, L.: Online Feature Extraction and Selection for Object Tracking. 
In: Proceeding of IEEE Conference on Mechatronics and Automation, pp. 3497-3502 (2007) 

17. Liang, D., Huang, Q.: Mean-Shift Blob Tracking with Adaptive Feature Selection and Scale 
Adaptation. In: IEEE Conference on Image Processing, vol. 3, pp. 369-372 (2007) 

18. Yin, Z., Poriki, F, Collins, T: Likelihood Map Fusion for Visual Object Tracking. In: IEEE 
Workshop on Applications of Computer Vision, pp. 1-7 (2008) 

19. Thomas, S., Gabriel, K.: A Quantitative Theory of Immediate Visual Recognition. Programe 
of Brain Research 165, 33-56 (2007) 

20. Jhuang, H, Serre, T, Wolf, L., Poggio, T: A Biologically Inspired System for Action Recog- 
nition. In: IEEE 1 1th International Conference on Computer Vision, vol. 14, pp. 1-8 (2007) 

21. Bar, M., Kassam, S.: Top-down facilitation of visual recognition. Proceedings of the National 
Academy of Sciences 103, 449^154 (2006) 

22. Yong, Y, Croitoru, M.: Nonlinear multiscale wavelet diffusion for speckle suppression and 
edge enhancement in ultra sound images. IEEE Transactions on Medical Imaging 25, 297- 
311 (2006) 

23. Sharifi, M., Fathy, M., Mahmoudi, T: A Classifed and Comparative Study of Edge Detection 
Algorithms. In: International Conference on Information Technology:Coding and Comput- 
ing, pp. 117-120(2002) 

24. Tan, X., Triggs, B.: Fusing Gabor and LBP Feature Sets for Kernel-Based Face Recognition. 
Analysis and Modeling of Faces and Gestures, 235-249 (2007) 

25. Aroussi, E., Amine, A., Ghouzali, S.: Combining DCT and LBP Feature Sets For Efficient 
Face Recognition. In: ICTTA 2008, pp. 1-6 (2008) 

26. Ojala, T, Pietikainen, M., Harwood, D.: A comparative study of texture measures with clas- 
sification based on feature distributions. Pattern Recognition 29, 51-59 (1996) 

27. Collins, T, Zhou, X.: An open source tracking testbed and evaluation website. Presented at 
the IEEE International Workshop on Performance Evaluation of Tracking and Surveillance 
(2005) 



Modelling and Performance Enhancement of a Linear 
Actuation Mechanism Using Conducting Polymers 



Elise T. Burriss 1 , Gursel Alici 2 , Geoffrey M. Spinks 1 ' 2 , and Scott McGovern 1 

1 Intelligent Polymer Research Institute 

2 School of Mechanical, Materials and Mechatronic Engineering 

University of Wollongong, 2522, NSW, Australia 

(etb984, gursel , gspinks, scottmg}@uow. edu. au 



Abstract. In this paper, we report on our investigation into modelling and 
performance enhancement of a linear actuation system based on cantilevered- 
type conducting polymer actuators, which can operate in air and aqueous media. 
We have employed the model to predict the linear displacement and force 
output of the actuation system, and to determine the optimum values of the 
system design parameters. The linear actuation system is a five bar parallel 
mechanism, articulated with two polymer actuators. Kinematic and force 
analyses of the mechanism including numerical results are presented, and its 
payload handling ability was experimentally evaluated. The experimental 
results prove that it is possible to generate an accurate linear movement and a 
corresponding rectilinear force from this mechanism. This mechanism can be 
employed as a motion and force transmission mechanism, which not only has a 
light weight, but also consumes a small electrical power. 

Keywords: Electroactive Polymer Actuators, System Optimization, Linear 
Actuation, Kinematic Design. 



1 Introduction 

Conducting polymers are favourable as actuators due to their low actuation voltage, 
high force output relative to their weight, good strain properties, light weight, simple 
structure and silent motion. The common materials used for actuators are Ionic 
Polymer Metal Composite (IPMC), and electroactive polymers (EAPs) such as 
Polyp yrrole (PPy) and Polyaniline (PANI) [1]. Disadvantages of polymer actuators 
include low speed of response, cyclic fatigue and non-linearity, which increases 
difficulty in modelling and controlling the actuator [2-3]. 

Bending actuators have been successfully used in applications where a nonlinear 
motion is required, such as a fin in an artificial fish [4]and robotic fingers [5], 
However, the bending motion may limit the control of actuators along a linear path. 
Conducting polymers have been used as actuators in applications such as medical 
devices, toys, digital camera accessories and artificial muscles for robots as developed 
by Eamex [3], Various PPy linear actuator designs have been investigated, including 
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were connected in parallel and immersed in a LiC10 4 aqueous solution, were able to 
produce a displacement of 60% of the actuator length in the longitudinal direction of 
the actuator. The use of linear actuators in air may provide an alternative when 
encapsulation of the electrolyte is not feasible. Yamakita et. Al [7] have developed an 
IPMC linear actuator for use in a biped walking robot. PPy actuators are used in 
applications where the oxidised or reduced position is maintained under a constant 
voltage, for which IPMC is not suitable as it will return to its original position. To 
overcome this limitation of IPMC, Thompson [1] adopted the design by Yamakita et 
al. using polypyrrole film as the active layers of a multi-layer actuator structure. Two 
methods of connecting the PPy film together to produce the linear design were 
employed by Thompson: (i) masking part of the film during PPy growth to create a 
flexible, electrically conductive hinge; (ii) electrically connecting the film with Cu 
tape, held in place with NdFeB magnets. This study implements the design 
investigated by Thompson using the second method of connecting the film together, 
as the method of masking allows the actuator to produce a clover-like shape under 
actuation, which impedes control and stability of the actuator. However, limitations 
arising through the use of Cu tape may include oxidation of the Cu, converting the 
tape to an insulator, thus limiting the actuation of the PPy components connected by 
copper tape. For optimisation of the PPy film in this study, the geometry of PPy 
bending actuators is limited to 50um thickness and 4mm width as the charge 
distribution has been found to be no longer uniform at thicknesses greater than 50- 
60um and curling is prominent at widths greater than 4mm, producing an increase in 
elastic stiffness of the actuator [2, 8]. These limitations are also expected to be evident 
in PPy linear actuators. 

A significant amount of work to determine the performance of bending PPy 
actuators has been previously conducted [5, 8] However, the complexity and greater 
degree of variability in a linear actuator means that the performance of a bending 
actuator may not directly translate to linear actuators. The modelling of linear 
actuators is necessary to accurately predict the force, displacement and work outputs 
for control of the actuator in practical applications. Modelling and optimisation work 
on PPy bending actuators by Alici, Metz & Spinks [2], in which a mathematical 
model to calculate the expected bending behaviour of PPy bending actuators was 
developed, and suggests that as the length of the actuator decreases, force output 
increases. This study has investigated the effect of the length of PPy components in 
linear actuators to determine if length has a similar effect on outputs in linear 
actuators. Combinations of 2mm, 3mm, 4mm wide and 10mm, 15mm and 20mm 
long linear actuators, where the length refers to the length of each component, were 
experimentally evaluated for force and displacement outputs. It was found that an 
increase in the length of the PPy links in the linear actuators allowed a greater range 
of motion, whilst shorter and wider actuators lifted greater loads. The linear actuators 
were also compared to bending actuators of corresponding dimensions and it was 
found that the bending actuators had a greater range of motion and the linear actuators 
lifted heavier loads. As the proposed linear actuation mechanism converts the 
rotational work into the linear work, ideally they are equal to each other, it is expected 
that while the displacement is decreasing, the force is increasing, and vice versa. 
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2 Functioning Principle of Linear Actuation System 

As the polymer actuators considered in this study are cantilevered from one end, they 
generate a rotary (bending) type motion. This is analogous to a single jointed crank 
motion. A single degree of freedom mechanism such as a four-bar mechanism does 
not allow control the output motion. With this in mind, a five-bar mechanism which 
needs two inputs (two bending type polymer actuators) is one of the multi crank 
mechanisms having practical importance, especially for following any arbitrary 
trajectory precisely [9]. This mechanism serves as a bending to-linear motion 
converter; converting bending angle into a linear movement, and bending moments 
into linear forces. 

2.1 Kinematic Analysis 

The topology of the mechanism is depicted in Figure 1, where it is assumed that the 
mechanism links are rigid for the sake of generating a pseudo-rigid-body-model 
for the size and operation optimization of this linear motion mechanism. The joints 
connected to the frame are the active ones. The others are passive. For the joint 
inputs 0j and 2 and the specified link lengths L ,L,,L 2 ,L 3 ,L 4 the analytical 
expressions for the coordinates of the output point P are obtained using the following 
algorithm [9]; 



L, =L, cosG, i +L, sinGj j, 

L 5 = (L + L 4 cos 2 ) i + L 4 sin 2 j, 



(1) 



AB|=R=|0 1 B|-L 1 =Ci+Dj, (2) 

(3) 



C=O.B -L, , D=0,B -L, , 

I i lx i* I i ly iy ' 



R = VC 2 +D 2 , 



Q = — - 2 L , (4) 

2R 2 



The coordinates (x p , y p ) of the output point P are; 



,+QC-dJ^-Q 2 



=>\+QD + C 1 fc--Q 2 



(5) 



R 2 

Depending on the link lengths, the expression in the square-root can become 
negative, which suggest that some kinematic design constraints are not satisfied. For a 
linear motion output, the output point can be constrained to move in a rectilinear 
fashion. This requires that 0, = 2 AND L, = L 4 AND L 2 = L 3 . Such a mechanism 
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P{x p ,y P ) 




Fig. 1. Proposed linear actuation module based on a parallel five-bar mechanism 



will convert all of its rotary work into the linear work - an efficient motion converter. 
The analytical expressions for the output point of such a linear motion module are 
obtained as 

x p =-L, cosG, + — [L +L,(cos0 2 +cos9,)J 



-L^sinGj -sinG,) 



R 1 



y p =Lj sinGj + — -(sin0 2 -sin0,) 



■[L + L,(cos0 2 +cos0j)] 



R : 



(6) 



where 

R 2 = L 2 o +2L o L,(cos0 2 +cos0,) 
+ 2L 2 [l + cos(0,+0 2 )] 

Depending on the link length of the mechanism, Eq.6 simplifies to 

i) ForL, = L 2 and0, =0 2 => x p =— °- and 

y p =L, sinG, 

+ -t]4L] -L 2 -^L^cos©, -4L 2 cos 2 0, 



(7) 
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ii) 



iii) 



For 



:0 



and 



0, - 0, 



= 



and 



y p =L, sin0, +^/L 2 2 -L] cos 2 0, 

For L =0, L, =L 2 and 0, =0 2 => x p =0 and y p = 2L, sin0, 



When choosing the link sizes, it is important for effective force transmission to 
minimize the variation of the transmission angle \\f from 90°; the acceptable range is 

90° + 40° [10]. It is mathematically expressed as 



cos \|/ = 1 - 



1 



'R V 



v L ,y 



(8) 



For 50° <\|/<130°, 0.845 < 



'r^ 



V^2 J 



< 1.8126 . With reference to Eq.8, for a given L 



and Lj , the range of L 2 satisfying the effective transmission angles can be calculated. 
It must be noted that L should be greater than L, to prevent any physical 



interference among the mechanism links. A practical ratios of 1 .5 < 



'L > 



V^i j 



< 3.0 and 



1.0 < 



'l ^ 



v L w 



< 2.0 should result in proportionate link lengths and transmission angles. 



For the data shown in Table 1, the transmission angle and the vertical movement of 
the linear actuation module are calculated and presented in Figure 2. 

Table 1. Numerical values of the parameters for the exemplary results shown in Figure 2 



Parameters 


L =40mm 




L 


= L 3 = 20 mm 




L 


= L 4 =22mm 


9 


— 


0, =90° to 115° 



The corresponding generalized relationship between rate of change = \d x 0, ] 

of the input bending angles and the output velocity vector X = [x P y P ] is given by 
[11]. 



X: 



3x„ 



3y F 



a(0„0 2 ) a(e,,e 2 ) 

where J is the mechanism Jacobian matrix. 



= 10 



(9) 
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Fig. 2. Variation of the vertical position of the output point P and the transmission angle with 
the input bending angles. The net vertical distance is 7.6858 mm. 

2.2 Force Analysis 

Assuming that two active links 0,A and 2 B made of electroactive polymers 
generate bending moments M , and M 2 , which act on the actuation module in the 
opposite directions. Assume that the output point P can apply a planar force vector of 
F = F i + F j to the environment to realise a functional task. Please recall that the 

mechanism converts the work in the bending coordinates into the work in linear 
coordinates X. These two works are ideally equal to each other. 
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Fig. 3. Variation of the vertical force component of the output point P with the input bending 
angles. The net vertical force is 0.6837 mN. The horizontal component of the force is zero. 



Using the duality between the generalized relationships for motion and force 
transfer between the actuation and output spaces, the following force relationship is 
obtained [11]; 
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M = J T R 



-} T M 



(10) 



where F p = [F F v J t and M 
in Table 1 and M = [l l] 



^M, M,J t and (.) T denotes transposition. For the data 



Nmm, the force output at point P is calculated using 
Eq.10 and is presented in Figure 3. Because the mechanism link lengths are chosen 
such that it can generate a vertical movement, the mechanism will not create any 
horizontal force component. These kinematic and force analyses suggest that it is 
possible to optimize the topology of the mechanism for an efficient motion and force 
transmission. 



3 Conducting Polymer Actuators: Synthesis and Actuation 
Principle 

The PPy film was manufactured by combining a solution of 0.1M pyrrole monomer 
with 0.1M lithium triflouromethanesulfonimide (Li + TFSF) in propylene carbonate 
(PC) with 1 % H 2 0, degassed with nitrogen gas and stirred for 15 minutes. This 
solution was used to grow the PPy onto a gold sputter coated, 0.45(J.m Millipore 
Immobilon-P porous poly(vinylidene fluoride) (PVDF) film by electrodeposition over 
a period of 12 hours at -33°C with a current density of O.lmA.m". After 
polymerization, the film was washed in acetone and then resoaked in a 0.1 M LiTFSI 
electrolyte and then stored away from oxygen. Actuators were then cut to desired size 
using a scalpel and were limited to size by the area of the 40mm x 55mm film. The 
resulting film consists of two PPy outer layers and two gold layers separated by a 
PVDF core is shown in Figure 4. 



PVOF 



Fig. 4. General structure of trilayer polymer actuator 



After doping with electrolyte, both the PPy layers of a bending actuator become 
partially oxidised (Figure 5a) and upon application of a positive voltage, one of the 
polymer layers is highly oxidized and the other is reduced. In an attempt to neutralise 
the charge imbalance, ions transfer from the electrolyte into the polymer layers. In 
LiTFSI doped actuators, TFSI" anions move into interstitial spaces in the polymer 
backbone of the oxidized layer, causing it to expand. Simultaneously, the opposing 
polymer film is reduced and contracts by removing TFSI" ions, overall creating a 
bending motion (Figure 5b.) By applying a square wave voltage, the actuator is 
allowed to return it to its original position and continue to bend through the neutral 
position so that the actuator may bend alternately in both directions. 
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LiTFSI in PVDF 



Au 



partially oxidised 




\ + 

contacts oxidised 



Fig. 5. Schematic of PPy actuator doped in LiTFSI (a) partially oxidised (b) under an applied 
voltage 

To produce the linear actuators, two pieces of PPy film cut to equal dimensions 
were connected together in parallel at their tips with copper tape and two 3mm x 
0.5mm neodymium iron boron (NdFeB) magnets one either side of each tip (Figures 
6-7.) Once placed between the electrodes, the PPy configuration effectively produces 
four PPy components, with the length referred to throughout this paper as the length 
depicted in Figure 7b. 

Under an applied voltage, it was observed that the PPy linear actuator produced the 
shape as shown in Figure 8. In this case, the outer PPy layer of the upper components 
is oxidised and the inside layer is reduced, and under no constraints, almost uniform 
curvature may be experienced. The lower components may be activated; however the 
possible oxidation of the Cu tape suggests that the lower components remain either 
inactivated. This linear design restricts the bending motion of the actuators due to the 
presence of the magnets used to clamp the copper tape to the actuators (Figure 8). For 
the shorter lengths, a greater percentage of the actuator is affected by the clamping 
induced by the magnets and as a result, the actuator has a more elongated appearance 
(Figure 9). Due to this elongation, it doesn't move as far in the vertical direction as it 
potentially could. To overcome this limitation, another method of electrically 
connecting the upper and lower actuator components that does not restrict the motion 
is necessary. For this to be possible, the electrical connection must be flexible and 
able to bond to the actuator without the assistance of magnets. Although the magnets 
restrict motion, they do provide some stability which is important in attempting to 
control the actuator. 



Pt electrodes 
NdFeB magnet 



PPy actuator 




Cu taps 



Fig. 6. Configuration of the linear PPy actuator with Cu tape and NdFeB magnets 
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Fig. 7. Linear PPy actuator in neutral position 

4 Performance Characterization Results 

Linear actuators shown in Figures 6-7 with link lengths of 10mm, 15mm, 20mm and 
widths of 2mm, 3mm, 4mm were manufactured and tested under a series of applied 
loads. Loads were added to the bottom tip of the actuators and the overall 
displacement of the base tip was measured visually using a grid paper, as illustrated in 
Figure 10a. The initial load used was the non-magnetic wire hook and a 0.266g 
NdFeB magnet glued to its base (Figure 10b), with incremental loads applied by 
adding magnets of either 0.266g or 0.54g to the base of the wire hook. Voltage was 
maintained at +/-1V to avoid over-oxidation and reduce the variation in results that 
may arise due to early onset of fatigue. The frequency of the square voltage inputs 
was held constant for all experiments at 0.5Hz to allow sufficient response time. 
Typical voltage, current and charge data recorded by the datalogging system used is 
presented in Figure 1 1 . 
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Fig. 8. Shape of actuator under actuation 
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Fig. 9. Shape of shorter actuators under actuation 
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Fig. 10. (a) PPy linear actuator with a load, (b) Wire hook; used to separate magnet loads from 
magnets on the actuator. 

Figures 12(a)-(i) provide an overview of the base tip position in extended and 
contracted positions under given applied loads, with the electrode contacts as the 
reference point. In all actuators, as the load increases the position of the base tip 
below the reference point also increases. Generally, as the width decreases the 
position below the electrode increases and is more evident in the 15mm long 
actuators. As width increases, the actuator becomes stiffer which allows a smaller 
deflection below the reference point under a load than an actuator of a small width. 

Displacement increases and then decreases for increasing load as shown in Figure 
12(d)-(i) for the 15mm and 10mm long actuators. This behaviour is because under no 
load, the actuators are restricted in their movement by the magnets clamping the Cu 
tape in place. The clamping effect has a greater impact on shorter actuators because, 
as a percentage, a greater proportion of the actuator is restricted than longer actuators. 
Under applied loads, the shorter actuators are able to move because the load forces the 
actuator base tip to be displaced to lower than the original position, and under an 
applied voltage the actuator attempts to return to the original position and produces an 
upward motion. For this reason, the shorter actuators may only be suitable for 
applications requiring a pulling load, and if shorter actuators are required for pushing, 
then another method of connecting the actuators that provides less restriction on 
motion is required. 
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Fig. 11. Example voltage, current and charge input data for 20mm x 2mm linear PPy actuator at 
0.5Hz 
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Fig. 12. Position of base tip of linear actuators under applied load 
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Fig. 12. (continued) 
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Fig. 13. Variation of the tip displacement of the linear actuation module with the payload 



A comparison of load and displacement output between the linear and cantilever 
actuators of corresponding length has revealed that the cantilever actuators provide a 
greater range of motion in the vertical direction, while the linear actuator lifted 
heavier loads. The linear actuators are effectively 4 times the volume of their 
corresponding bending actuator. However, they produced much less displacement due 
to the restriction in motion provided by the magnet connections. The cantilever 
actuators have the ability to move above and below the electrodes whereas the linear 
actuators are limited to motion below the electrodes only. Although much less 
displacement is achieved, the benefit of a linear actuator is that motion in one 
direction is more controlled than the motion of a bending actuator. For displacement 
output, the linear actuators with 20mm PPy links produced the greatest displacement 
than actuators with shorter PPy links over a range of loads. Figure 13 depicts the 
displacement in the vertical direction for linear actuators of varied width with 20mm 
length. As width of the PPy links increases, a greater displacement output for a given 
load may be expected. However, as width increases, the PPy film has a tendency to 
curl across the width, particularly in actuators for 4mm wide or greater [4]. Curling 
increases stiffness and thus decreases the potential displacement output, hence the 
4mm wide actuator produced a slightly smaller displacement than actuator with 3mm 
wide PPy links. 

The calculated work output for actuators of varying length and width is presented 
in Fig. 14 and compared by the width for each actuator length. For the 20mm long 
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actuators, the 3mm and 4mm wide actuators produced a similar work output, and 
were significantly higher than the 2mm wide actuator. As width increases, the 
actuators should experience more charge and therefore should produce greater work. 
However, the 4mm wide actuator may have experienced curling, in which it would 
have contributed some of its energy to curling across the width of the actuator. The 
curling effect may also impede motion by increasing the stiffness of the actuator, 
therefore reducing work output by restricting motion. 
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Fig. 14. Work versus load for PPy linear actuator: (a) 20mm long components (b) 15mm long 
components (c) 10mm long components 



Work versus load plots should depict a parabola shape, which is evident in the 
20mm long actuators (Fig. 14a) The parabola shape is not present in the work versus 
load plots for the 15mm and 10mm long actuators (Fig. 14b-c) because a significant 
proportion of work being done was being opposed by clamping of the magnets at the 
hinges. Therefore, the magnets impeded the work output for shorter actuators, and the 
development of an alternate method of joining the components may allow greater 
output for shorter actuators. Maximum work for the 4mm x 15mm and 4 mm x 
10mm actuators is not depicted in the Fig. 14b-c, however at the maximum work 
measured, these actuators produced very little displacement (Fig 13.) These results 
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suggest that the increased stiffness of the actuator created by the curling effect of the 
actuator could be useful in applications where higher loads and very small 
displacements are required. 

Work density can be used to compare actuators of different geometries, 
performance and efficiency and is also a good measure to determine the actuators that 
are not performing near their potential. Fig. 15 compares the work density versus load 
of the 20mm long actuators with varying width. For these configurations, the 3mm 
wide actuator shows a significantly higher work density than both the 2mm and 4mm 
wide actuators. The 2mm and 4mm actuators exhibited similar work densities, and 
further testing would be required to determine which is more efficient. The 2mm 
actuator has a similar density to the 3mm wide actuator at lower loads (<lmN), but 
has significantly lower work density at higher loads. The 4mm actuator is less 
efficient than the 3mm actuator, due to a proportion of the input energy being 
contributed to curling across its width. Because of its higher work density, the 3mm 
wide actuator would be more preferable when constraints such as cost and volume 
exist. However, because the 2mm and 4mm wide actuators have a lower work density 
than the 3 mm wide actuator, there is greater potential for their work output to be 
improved. If the efficiency of the 4mm wide actuator was improved then it would 
have a higher work output than the 3mm, which is shown in Figure 14. 
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Fig. 15. Work density of PPy linear actuator with 20mm long components 



5 Conclusions 



A linear actuator system based on the bending motion of conducting polymer 
actuators operating in air is presented, including an analytical model to estimate the 
linear movement and the force output of the mechanism. The mechanism is basically 
a motion and force transmission system, converting the bending work provided by the 
electroactive polymer actuators into Cartesian work. The experimental results 
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presented demonstrate that the conducting polymer actuators generate enough 
displacement and force to handle a range of practical payloads. Another outcome of 
this study is that when the bending type- low power consuming polymer actuators are 
tailored properly, they can be used to generate a rectilinear motion with enough force 
output. 

Future work involves deriving a more accurate analytical model taking into 
account the deflections of the mechanism links and verifying the model 
experimentally. Improvements may also be made to the hinge connections of the 
linear actuator, by replacing the copper connections with an inert, conductive material 
such as gold or platinum. 
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Abstract. Due to the increased complexity of tasks delegated to unmanned 
systems their collective use is becoming of paramount importance for performing 
any reasonable jobs. An approach is offered where integral group behaviors are 
achieved automatically rather than set up manually. Missions in a high-level 
Distributed Scenario Language can be executed jointly by communicating 
inteipreters embedded in key system units. Robotic scenarios like reconnaissance, 
camp security, convoy, mule and explosive ordnance disposal oriented on 
different numbers of cooperating units are demonstrated. The approach allows 
us to effectively manage any teams, from manned to purely unmanned, regardless 
of the number of components in them. Other researched applications of the 
technology are outlined as well as its relation to gestalt philosophy, where the 
whole dominates over parts, defining their sense and even existence. The paradigm 
discussed may also be considered as a universal globally programmable distributed 
super-machine operating with both information and physical matter, ruling the 
world covered by it at runtime. 

Keywords: Unmanned Systems, Distributed Scenario Language, Networked 
Interpretation, System Integrity, Robotic Swarms, Reconnaissance, Camp 
Security, Convoys, Explosive Ordnance Disposal, Gestalt, World Super- 
machine. 



1 Introduction 

With the world dynamics increasing due to global warming, numerous natural and 
manmade disasters, military conflicts, and international terrorism, using unmanned 
(ground, sea, underwater, and air) systems can alleviate many problems and save lives 
in hazardous environments. Because of the complexity of tasks delegated to 
unmanned solutions and still insufficient capabilities of existing robotic vehicles, their 
simultaneous, collective use may be of paramount importance to perform any 
reasonable jobs. Operating together, the unmanned groups, often called swarms, can 
fulfill the required objectives despite possible runtime damages to individual units. 

We are offering a novel approach to organization of unmanned systems, oriented 
from the very beginning on parallel solutions in physical spaces, with swarm 
behaviors resulting naturally and accomplished automatically, rather than 
programmed manually. This approach, symbolically called "overoperability" from the 
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previous publications [1,2], allows us to create, modify, analyze, process, and manage 
any distributed systems, establishing local and global dominance over them. 

Within the overoperability philosophy, an integral mission scenario, written in a 
special high-level scenario language [2,3] and reflecting semantics of what to do in a 
distributed space rather than details of implementation, is executed in a parallel and 
cooperative manner by dynamically networked unmanned units. During the scenario 
evolution, any operations can be accomplished in the world, along with the needed 
movement of code, equipment and "doers" (both artificial and biological) and 
creation and maintenance of physical and virtual infrastructures supporting the 
missions. 

This paper has been essentially inspired by the European Land Robotic Trial [4] in 
which the author participated. It was conducted to provide trials as close as possible to 
operational scenarios for UGVs/UAVs with focus on short-term realizable robot 
systems. The day and night trials were organized within the following five main 
scenarios: non-urban reconnaissance, camp security, transport convoy, transport mule, 
and explosive ordnance disposal. Only a limited number of robotic units were 
engaged in every scenario, just one or two, whereas every scenario could potentially 
be executed with much higher efficiency if using robotic teams with many units, 
which cooperate with each other. 

The paper also reflects activity on the project started under the sponsorship of 
Alexander von Humboldt Foundation (AvH) in Germany. One of its aims is 
formalization of known mission scenarios in such a way that they could be performed 
by any available numbers of robotic vehicles, with the management burden 
effectively shifted to self-organized robotic teams — thus relieving human operators 
from traditional routines and allowing them concentrate on mission goals and global 
efficiency instead. 

2 Distributed Scenario Language (DSL) 

The approach described here is based on the Distributed Scenario Language (DSL) 
which allows us to set what to do in a distributed world on a semantic level, 
abstracting from details of how to do this and with which resources, delegating these 
to the intelligent automatic interpretation. Being a universal programming language 
with advanced parallel and distributed capabilities, DSL can also describe tasks and 
behaviors on any levels and their mixtures, if needed. The language can be used by 
humans who should follow its instructions individually or collectively, or can be 
directly executed by robots and their teams. Any mixed human-robotic organizations 
can be managed in DSL too. 

2.1 DSL Worlds 

The worlds DSL operates with can be virtual, physical, or combined. 

■ Virtual World (VW) is discrete and consists of nodes and links connecting these 
nodes. Any information can be associated with both nodes and links in the form of 
their names (contents). Nodes have unique addresses in VW, whereas their names 
(same as names of links) may repeat throughout the VW. Nodes can be accessed 
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directly, globally, by their names or addresses, or locally, from each other, via the 
(named) links, whereas links can be accessed only locally— from the adjacent 
nodes. A variety of broadcasting possibilities are available in the VW, both in a 
global and local way, for example, from outside to all nodes, from a node directly 
to all other nodes, or from a node to all neighboring nodes via the selected or all 
adjacent links. 

■ Physical World (PW) is continuous. Any point in it can be identified and accessed 
by the coordinates expressed in a certain coordinate system, also with certain 
precision. Staying in a PW point, you can lift local physical parameters from the 
world and, possibly, also change them, impacting the world locally too. 

■ Virtual-Physical World (VPW) is the one where VW nodes additionally associate 
with certain coordinates in the PW. VPW is discrete on a snapshot, but the nodes 
can change their physical coordinates overtime. The VPW nodes can be globally 
accessed by their names, addresses, or physical coordinates (for the latter, more 
correctly: by coordinates of the expected center and a radius of the region, due to 
limited precision of the coordinates). Also locally, via links— same as for the pure 
VW. In addition to the broadcasting capabilities of VW, nodes in VPW can also 
be massively accessed/entered by identifying a probable region in PW where they 
are expected to exist— by the region's center and a range (radius) from this center, 
where the latter may be of any value. 

2.2 High-Level Scenarios 

The world, as described above, is navigated and processed in a parallel and distributed 
way by high-level DSL scenarios having the following main features. 

General Features. These define runtime coverage, or grasping, of distributed spaces. 

■ A DSL scenario (or program, in a conventional notation) describes development 
of activities in a distributed world as parallel transitions between sets of progress 
points, or props. 

■ Starting from a prop, a program action may result in one or more new props, or 
remain in the same prop. 

■ Each prop has a resultant value and a resultant state. 

■ Different actions (whatever complex they might be), starting from the same prop, 
may evolve independently or interdependently, and sequentially or in parallel, 
each contributing to the resultant set of props on this group of actions. 

■ Actions may also succeed each other in the space of props, with new actions 
applied in parallel from all props reached by the previous actions, resulting 
altogether in the integrated set of props on all these applications. 

■ Elementary operations can be defined on the values of props reached by other 
actions (the latter of any complexity), leading to the resultant prop with associated 
value (which may be multiple) and resultant state. 

■ The scenarios can form new or remove existing nodes and links in the distributed 
VW or VPW, allowing us to create, modify, and process any graph-based 
infrastructures in these worlds. 
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Association with World Nodes/Points. Scenarios evolve directly in distributed 
worlds while matching them, where: 

■ Any prop can be associated with a node in VW or a position in PW, or both, like 
in the case of VPW. 

■ A prop can also be linked separately with VW nodes and PW positions, allowing 
us to operate with the two worlds independently. 

■ Any number of props can be associated simultaneously with same points of the 
worlds. 

■ Staying with nodes/positions in the worlds, a prop allows us to directly access 
local data in these points, both virtual (information) and physical (matter). 

Different Types of Variables. Used together, they allow us to create efficient spatial 
algorithms which work in between components of distributed systems, rather than in 
them. The variables are classified as:Heritable variables - these are starting in a prop 

and serving all subsequent props, which can share them in both read & write 

operations. 

■ Frontal variables - are an individual and exclusive prop's property (not shared 
with other props), being transferred between the consecutive props, and replicated 
if from a single prop a number of props emerge. 

■ Environmental variables - are accessing different elements of physical and virtual 
words when navigating them, also a variety of parameters of the internal world of 
DSL interpreter. 

■ Nodal variables - allow us to attach an individual temporary property to VW and 
VPW nodes; they can be accessed and shared by any props associated with these 
nodes. 

Hierarchical Control. The scenarios can be locally and globally controlled, with 
flexible watershed between the two options. 

■ DSL scenarios can use a variety of spatial control rules, allowing us to assess local 
and remote states, make local and global decisions, and invoke or skip subsequent 
and terminate current operations, on the results of these decisions. 

■ Nested control infrastructures, embracing the whole scenario, provide 
interdependent local and global decisions associated with points of the worlds. 

2.3 The Language Syntax 

DSL has a recursive syntax shown below together with the names of its main 
constructs (where square brackets are for an optional construct, braces mean 
construct' s repetition with a delimiter at the right, and vertical bar separates 
alternatives, whereas the whole program is historically called a wave [3]). 

wave -> constant I variable I [ rule ] ({ wave , }) 

constant -> number I string I special 

variable -> identifier I reserved 
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rule -> expand I transfer I modify I branch I advance I repeat I grant I echo I 

arithmetic I structural I assign I compare I timing I type I usage I identifier 
special -> abort I thru I done I fail I any I random I all lout I in I infinite I nil I empty I 

first I last random I virtual I physical I combined I neighbors I global I 

local I direct 
reserved -> Njalphameric} I Hjalphameric} I Ffalphameric} I TYPE I QUALITIES I 

NAME I ADDRESS I PLACE I WHERE I BACK I PREVIOUS I LINK I 

DIRECTION I ORDER I WHEN I TIME I SPEED I STATE I VALUE I 
COLOR I RESOURCES I DOER I USER I START 

expand -> hop I move I create I linkup 

transfer -> run I call I output 

modify -> split I partition I select I replicate I integer 

branch -> par I sequence I if I while I or I par or I and I par and 

advance -> advance I sync advance 

repeat -> cycle I loop I sling I repeat I repeat sync 

grant -> free I release I quit I none I lift I stay I grasp 

echo -> rake I min I max I sort I sum I average I product I count I state 

arithmetic -> add I subtract I multiply I divide I degree 

structural -> separate I unite I concatenate I append I intersect I content I index I rand 

assign -> assign I assign peers 

verify -> equal I not equal I less I less equal I more I more equal I empty I nonempty I 

belong I not belong I inside I not inside 

timing -> sleep I remain 

type -> nodal I heritable I frontal I environmental I info I matter I number I string 

usage -> address I name I place I center I range I time I speed I doer I node I link I unit 

The basic construct, rule, can represent any action, decision or a guideline, for 
example: 

■ elementary arithmetic, string or logic operation; 

■ hop in a physical, virtual, or combined space; 

■ hierarchical fusion and return of (remote) data; 

■ distributed control, both sequential and parallel; 

■ a variety of special contexts for navigation in space, influencing different 
operations and decisions; 

■ type or sense of a value or its chosen usage, assisting automatic interpretation. 

Different variants of this syntax and semantics had been implemented for previous DSL 
subsets [2,3], where conventional expression of operations and delimiters between 
program parts can be used too, for better readability and compactness. For example: 
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3 + 5 + 7 - is the same as add ( 3 , 5 , 7 ) , 

wl; w2; w3 - is the same as advance {wl, w2 , w3) , 

where wl to w3 may be arbitrary DSL programs (waves) themselves. The first 
example could have any programs instead of just numbers, each returning its 
(possibly, remote and multiple) results: 

wl + w2 + w3 - same as add (wl, w2 , w3) . 

3 Distributed Interpreter 

A variety of options may be available for automatic interpretation of DSL scenarios - 
from fully centralized and sequential to fully distributed and parallel. Due to peculiar 
syntax and semantics, the language interpretation in distributed systems is transparent 
and straightforward. Some basic features of the DSL interpretation are as follows. 

■ Direct association of props with world points drastically simplifies bringing data 
from the points to scenarios or vice versa: scenarios or their parts to world points. 

■ Chained actions can self-navigate and match the world, while omitting used 
"heads" and forwarding remaining "tails" further. 

■ Independent actions can be launched in parallel, developing autonomously in parts 
of the world. 

■ The interpreter copy can be installed in internet hosts, mobile robots, laptops, 
mobile phones, smart sensors, or implanted into biological units. 

■ The interpreter can also be a human being, performing manually of what is for 
herself while passing other parts of the scenario to other human or electronic 
interpreters and establishing dynamic command and control infrastructures 
between them. 

■ Any other systems can be accessed via the networked interpreters, the latter 
forming a supervisory layer managed in DSL. 

■ The interpreter copies may be concealed inside the systems to be impacted, even 
without their knowledge (to work in hostile environments). 

■ The interpreters can also migrate in the worlds to be managed, collectively 
executing (mobile too) mission scenarios, resulting altogether in a flexible and 
ubiquitous system organization. 

■ The DSL interpreter consists of a number of specialized modules working in 
parallel and handling and sharing specific data structures, which are supporting 
both persistent virtual worlds and temporary spatial control mechanisms [2,3]. 

■ The heart of the distributed interpreter is its spatial track system enabling 
hierarchical command and control and remote data & code access, also providing 
high integrity of emerging parallel and distributed solutions, achievable without 
central facilities. 

In application to robotic communities, the approach allows us to convert any group of 
mobile robots into a goal-directed cooperative system by integrating copies of the 
DSL interpreter, represented as a universal control module U in Fig. 1, with 
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traditional robotic functionalities (as, say, in [5]). (The figure exhibits mobile robots 
participated in the ELROB trial [4].) 




Fig. 1. Robotic teaming using embedded DSL interpreters 

Any mission scenario in DSL can start from any robot, covering and tasking the 
whole system (or its parts needed) at runtime and in parallel. Subordination between 
the units, and dynamic command and control are established automatically, as a 
derivative of the mission scenario and current state of environment. 

Due to fully interpretive nature of the technology, the scenarios can self-recover 
from any points, timely reacting on failures of robots. The whole group may remain 
fully functional and global-goal-oriented even in case of indiscriminate damages to 
individual units. 



4 Elementary Example 

An elementary task to be programmed in DSL may look like follows: 

Go to given physical locations of the disaster zone (represented in a proper system 
of coordinates by the three locations): (50.433, 30.633), (50.417, 30.490), and 
(50.467, 30.517). Evaluate damage in each location, then find and return the 
maximum destruction value on all locations to a management center. 
The corresponding program in DSL will be as simple as: 

maximum ( 
move) (50.433, 30 . 633 ) , (50 . 417 , 30 . 490) , (50 . 467 , 30.517)); 
assess (damage) ) 

This program reflects semantics of the task to be performed in a distributed space, 
regardless of possible equipment that can be used for this. The latter may, for 
example, be a set of sensors scattered in advance throughout the disaster zone, where 
hopping by coordinates may result in a wireless access to the sensors already present 
there, not necessarily moving into these points physically. 

As another solution, the program may order mobile robots to move into these 
locations and perform the needed damage assessment upon reaching the destinations. 
Some snapshots of the behavior of three available robots (Rl, R2, and R3), 
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cooperatively performing this task, are shown in Fig. 2. After the initial injection of 
DSL program into Rl (Fig. 2a), a distributed interpretation infrastructure covering all 
three robots is created, and the scenario is partitioned and spread between the robots 
via this infrastructure (Fig. 2b). 

All three robots then move independently to the locations automatically chosen 
(Fig. 2c). In each location reached independently by a corresponding robot, the 
damage assessment takes place, and all the results obtained are then being collected 
by Rl which finds the maximum value among the three ones on different locations. 
This final result will constitute the resultant value of the whole DSL program which 
will automatically be returned to the management center which remotely injected the 
DSL program into robot Rl (Fig. 2d). 

As can be seen from the examples above, a semantic level scenario describing what 
to do in the distributed space can be interpreted by robotic teams autonomously, and 
by different numbers of cooperating robots (we could use two or a single robot 
instead, in which cases some robots will have to move to and serve more than one 
location in a sequence). The number of robots can also vary at runtime, during the 
scenario evolution, as some may get damaged while others involved at any stage. 
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Fig. 2. Three-robot solution, a) initial scenario injection; b) infrastructure creation with tasking 
all robots; c) simultaneous robot movement; d) parallel damage assessment, finding and 
returning maximum. 



5 More Robotic Scenarios 

We will be using here the main scenarios that formed the basis of ELROB [4]. 
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5.1 Non-urban Reconnaissance 

For this scenario, it is supposed that a group of unknown vehicles is located in some 
distance in a non-urban area (defined, say, with the position of a center and area's 
radius), with security situation unclear there, so the reconnaissance should be done by 
robotic vehicles for not risking own personnel. The objective is to go to this target 
area and search for vehicles with specific characteristics. If found, they should be 
examined in detail, with their parameters collected and reported to the control station. 
The general situation is shown in Fig. 3a, where the reconnaissance facilities 
should first go to the target area (towards its center), observe the area by 
cameras/sensors to roughly locate most probable targets (by their size, for example). 
The next will be to move directly to these selected targets and sense and collect their 
detailed parameters, with sending the results to the control point where they are 
accumulated and analyzed. 
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Fig. 3. The reconnaissance scenario, a) moving to and searching the area; b) avoiding obstacles. 

Parallel Solution. This, in DSL, may allow us to use as many reconnaissance 
vehicles as possible (a single one including), potentially involving individual vehicles 
for each target identified, for their detailed examination. 

USER = (move (start) ; WHERE = center; 

Targets = recognize (radius, features); 

split (Targets) ; WHERE = VALUE; collect (size, type, speed)) 

Explicitly Sequential Solution. The following DSL program just details navigation 
and organization procedures to execute the reconnaissance scenario in a strictly 
sequential way, which may be useful for optimization of the use of a single vehicle 
only. 

move (start) ; WHERE = center; 

Targets = recognize (radius, features) ; 
loop(WHERE = withdraw (Targets, 1); 

Result &= collect (size, type, speed)); USER = Result 

Avoiding Obstacles. The movement to the target area and inside it may be 
complicated due to the presence of obstacles, as shown in Fig. 3b. The following DSL 
program, for the move from Start to Center, uses an external procedure 
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approach_or_stop to detect obstacles and stop to avoid collision, and the 
procedure suitable to find next suitable waypoint on the way to the destination, 
from which the move may continue. 



move(start) ; loop (approach_or_stop (center) ; 

no t_close (WHERE, center); WHERE = suitable (depth, 



center) ) ; 



5.2 Camp Security 

For the camp security scenarios, a defined urban area has to be monitored (think 
military camp) and this should be executed by robotic vehicles too, to minimize risk 
to human personnel. The objective is to detect and report irregularities in the area, like 
intruders, while acquiring their positions and imagery, and transmitting to control 
station. 

The general picture is shown in Fig. 4, where the camp units (numbered 1 to 6) are 
simultaneously patrolled by a number of robotic vehicles moving along the paths 
between and around the buildings. 
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Fig. 4. Camp security solutions, a) random movement; b) using predetermined routes. 



Distributed Campus Map. The proper routing of vehicles and resolution of possible 
conflicts between them (like collision avoidance) can be assisted by the creation and 
use of a distributed map of the campus area (just reflecting Fig. 4a) by the following 
DSL program (with node names reflecting X-Y coordinates of the crossings, and all 
links named r): 

create (#3_1; Fl = A; r#2_l; F2 = A; r#l_l; F3 = A; r#0_l; 
(r#0_2; r#l_2; r#F3 , (r#2_2; r#F2,(r#3_2; r#Fl) ) ) , 
(r#0_0; r#l_0; r#F3 , (r#2_0; r#F2 , (r#3_0 ; r#Fl)))) 

Random Movement. The next program organizes the duty performance by three 
parallel processes (which may be executed by three robots) using the created 
distributed map, with random choice of the next hop crossing and activation of the 
external service procedure move_check_report to analyze the local security 
situation while on the move. 
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hop(0_l, 2_2, 3_0); WHERE = CONTENT; 

repeat (or ( (hop (link (random) ) ; grasp (Mark == nil; Mark = 1); 
(hop (BACK); Mark) = nil; move_check_report (CONTENT) ) , stay)) 

Movement Via Predetermined Routes. If to use predetermined routes only, like the 
ones shown in Fig. 4b (one route using links named rl and another one r2), the 
collisions between robots can be avoided in full. These additional links rl and r2 in 
the campus map can be installed by the following DSL program: 



linkup ( 












(#0_2; rl#l_2; 


rl#l_l; 


rl#l_0; 


rl#0_0; 


rl#0_l; 


rl#0_2) , 


(#3_2; r2#2_2; 


r2#2_l; 


r2#2_0; 


r2#3_0; 


r2#3_l; 


r2#3_2) ) 



And two independent spatial processes navigating the campus via the new links 
(which may engage two robots) can be organized by the following parallel DSL code: 

(hop(0_l); Flink = +rl ) , (hop (3_0 ) ; Flink = +r2 ) ; 
WHERE = CONTENT; 
repeat (hop (link (Flink) ) ; move_check_report (CONTENT) ) 

Any imaginable combinations of different types of simultaneous movement through 
the camp (like those by predetermined routes and/or by free, random, wandering) with 
collision avoidance can also be easily organized in DSL. 

5.3 Transport Convoy 

Imagine there is a delivery for a camp located in some distance. The objective is to 
move at least two vehicles to the target location, where only the first one can be 
manned and the second should follow the route of the first one, on a certain distance 
from it. We will consider a fully robotic solution for such a convoy, with two and also 
any number of vehicles, where only the first vehicle knows (and follows) waypoints 
toward the target location, while others dynamically chaining with, and following the 
previous ones on the move. 

Two-unit Convoy. It is represented by the communicating Leader and Follower, 
where the first one defines its movement by a sequence of waypoints, and the second 
one, regularly requesting the Leader, moves to the positions previously occupied by it, 
while keeping a certain threshold distance. This is shown in Fig. 5, and by the DSL 
program that follows. 



Waypoints S 




Fig. 5. Two-unit convoy 
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move {start) ; 
(create (Leader) ; Waypoints = [wl, w2 , w3 , ...) ; 

loop (WHERE = withdraw (Waypoints, 1))), 
(create (Follower) ,- sling (Lcoord = (hopirange, any); WHERE); 

distance (WHERE, Lcoord) > threshold; WHERE = Lcoord)) 

Multiple-unit Convoy. A scenario for the convoy with any number of chained 
processes (to be materialized by robotic units) is described by the following DSL 
program and depicted in Fig. 6. For this case, only the first process is a pure leader 
and the last process a pure follower, while all other processes combine both 
functionalities, i.e. being followers for the previous processes and leaders for the 
subsequent ones. 

move (start) ; cycle(N < number; create (N += 1) ) ; 
(NAME == 1; Waypoints = {wl, w2 , w3 , ...) ; 
Loop(WHERE = withdraw (Waypoints, 1))), 

(NAME != 1; sling(Lcoord = (hop(range, NAME-1) ,- WHERE) ,- 
distance (WHERE, Lcoord) > threshold; WHERE = Lcoord)) 
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Fig. 6. Multiple-unit convoy 



5.4 Transport Mule 



Fir this scenario, there are two camps with a certain distance in between, and a cargo 
with a given weight should be transferred between the camps. We will consider here 
different possibilities to deliver the payload between the camps, using unmanned 
vehicles as "mules". 

Delivery in a Single Piece. This may be the case if cargo's weight allows it to be put 
on a single vehicle, as shown ion Fig. 7. 

Cargo 



Fig. 7. Single piece cargo delivery 

The related DSL program will be as follows: 

move ( Campusl ) ; frontal (Cargo) = "substance' 
move (Campus2) ; Store = Cargo 
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Shuttling between Camps. For this option, the process shuttles as often as possible 
between the two camps after partitioning the cargo into portions of the weight 
allowed, unless all the cargo is delivered, as shown in Fig. 8 and by the following 
program. 




Fig. 8. Shuttling delivery 

move ( Campusl ) ; frontal (Load) ; Cargo = "substance"; Limit = 50; 
loop (or ( (weight (Cargo) > Limit; Load = withdraw (Cargo, Limit)), 
(weight (Cargo) > 0; Load = Cargo)); 
hop (Campus2) ; Store += Load; hop ( Campusl ) ) 

Multiple, Parallel Delivery. For this case, different processes (vehicles) are 
considered to be independent from each other, each moving to the destination as 
quickly as possible on its own (see Fig. 9 and the following program). 



Fig. 9. Parallel cargo delivery 

move ( Campusl ) ; frontal (Load) ; Cargo = "substance"; Limit = 50; 
cycle (or ( (weight (Cargo) > Limit; Load = withdraw (Cargo, Limit)), 

(weight (Cargo) > 0; Load = Cargo))); 
move (Campus2) ; Store += Load 

Multiple, Convoy Delivery. For this scenario, the vehicles, each with a limited 
partition of cargo, are dynamically chaining in a column for a cohesive movement 
towards the destination (see Fig. 10 and the subsequent DSL program). 

'r,™n,,^TN Limit Limit Limit Limit 

Campusl^} ►o yo ► o — ► ■■■ — 

Fig. 10. Delivery in a convoy 

move ( Campusl ) ; frontal (Load) ; Cargo = "substance"; Limit = 50; 
cycle (or ( (weight (Cargo) > Limit; Load = withdraw (Cargo, Limit)) 

(weight (Cargo) > 0; Load = Cargo)); create(N += 1)); 
(NAME == 1; move (Campus2) ) , 

(NAME != 1; loop (not_close (WHERE , Campus2) ; 
WHERE = (hop (NAME - 1); WHERE))); 

Store += Load 
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5.5 Explosive Ordnance Disposal 

Explosive Ordnance Disposal (EOD) means the detection, identification, onsite 
evaluation, rendering safe, recovery, and final disposal of Unexploded Ordnance 
(UXO) including detonation and burning. It is often said that the EOD operation is a 3 
Ds one, which is Dangerous, Dirty and Demanding (or Difficult) job. Using robotic 
vehicles, especially multiple ones, is therefore becoming the most promising EOD 
option. Various kinds of EOD scenarios for navigation and examination of the target 
territory may be offered. We will just hint here on the simplest two options, easily 
expressible in DSL. 

Sequential Territory Search. This represents a single-threaded process (oriented on 
a single vehicle), where the whole territory is incrementally scanned unless all being 
searched, as described by the following program and depicted in Fig. 1 la. 



Finish 



Y2 Finish 




X1 Y1 X2 X1 Start 



a) b) 

Fig. 11. Sequential territory search, a) conventional; b) zigzag-like. 



XI =...; X2=..., 


Y = Yl 


= ...; Y2 =. 


.; DY =.. 


; 


loop (WHERE = 


= (XI, Y) ; 


WHERE = 


(X2,Y) ; 


(Y += DY) < Y2; 


WHERE = 


= (X2,Y); 


WHERE = 


(XI, Y) ; 


(Y += DY) < Y2) 



The sequential coverage of the territory can be organized with minimum waypoints to 
pass, in a zigzag way, as shown in Fig. 1 lb and by the following program. 



XI =...; X2=.. 
loop (WHERE 
WHERE 



Y = Yl =...; Y2 
(XI, Y) ; (Y + = 
(X2, Y) ; (Y + = 



=...; DY =...; 

DY) < Y2; 
DY) < Y2) 



Parallel Territory Search. This can be represented by a number of independent 
processes, each starting from a different location, and navigating altogether the whole 
region in parallel, as depicted in Fig. 12a, and explained by the DSL program that 
follows (all processes follow the predetermined routes for this case). 
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Fig. 12. Parallel territory search, a) with predetermined routes; b) random movement. 



XI =...; X2=... 


; Yl =...; Y2 =...; DY =.. 


; frontal (Y) = 


Yl; DDY = 0; 


cycle ( (Y += 


DDY)<Y2; DDY += DY) ; 


WHERE =(X1,Y) 


; WHERE =(X2,Y) 



Parallel search of the territory can also be organized in a random way, where each 
process chooses its next hop randomly, also taking into account that the chosen next 
destination was not visited before (at least looking like this, say, with the help of 
visual sensors detecting prints after wheels). Parallel random search may have an 
advantage before predetermined search in that it can eventually cover all the territory 
despite possible failures of individual processes (robots). Such a search, with 
processes starting from some initial points (named cl to c5), also keeping certain 
threshold distance from each other, is shown in Fig. 12b and by the following 
program. 



move ( cl , c2 , c3 , c4, c5) ; Range 
XI =...; X2=...; Yl =...; Y2 =...; D =...; 

New = WHERE + 2 (random(-D, D) ) ; 

hop (New, Range) & seen (New) == 



loop ( 

inside (New, (XI, X2 , Yl, Y2 ) ) 
nil; shif t_check_act (New) ) 



6 Other Applications 

Many other applications of the paradigm are possible, as follows, some of which 
already investigated, tested, and results published [1-3, 6-10]. 

Emergency Management. Using interpreters installed in massively wearable devices 
may allow us to assemble workable systems from any wreckage after the disasters, 
using any remaining communication channels, manual including. These emergent 
systems can provide distributed self-awareness, collect statistics of casualties, guide 
the delivery of relief goods, and coordinate collective escape from the disaster zone. 

Directed Energy Systems. The technology can provide high flexibility in organizing 
directed energy (DE) systems, especially in crisis situations, making automatic 
distributed decisions with the "speed of light" too. It may also help automate the 
global power dominance by optimized delivery of directed energy into any world 
points via dynamically organized networks of relay mirrors. 
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Distributed Avionics. Implanting interpreter copies into main control points of the 
aircraft may provide a higher, intelligent, layer of its self-analysis and recovery, by 
the spreading recursive scenarios starting from any point and collecting & fusing key 
data from other points. The embedded interpretation network with local, dynamic, and 
emergent links will be fully functional under any damages, especially with wireless 
communications between the interpreters. This may always provide global control 
integrity, even in a physically disintegrating object, helping to save lives and 
complete missions. 

Sensor Networks. Wireless sensors may be dropped from the air massively, as "smart 
dust". Having a limited communication range, they must operate in a network to do 
nonlocal jobs in a distributed environment. With the technology offered, we can 
convert their emergent networks into a universal parallel computer operating in DSL. 
It can effectively solve complex distributed problems— from just collecting and fusing 
scattered data to outlining and assembling images of the distributed phenomena like, 
for example, flooding, smog, flocks of birds, movement of troops, etc. 

Advanced Command and Control. In DSL, it is possible to define high-level scenarios 
concentrating on mission goals and top decision-making while delegating C2 routines, 
appearing at runtime as a derivative of the mission and environment states, to 
automatic interpretation. It is also convenient to express in DSL any theoretical and 
practical issues of advanced C2 explicitly. 

Infrastructure Protection. Navigating the systems at runtime, the technology can 
analyze safety and integrity of critical infrastructures and key resources, establishing 
protective networked mechanisms throughout them. Other systems can be involved 
from the DSL interpretation layer for emergent infrastructure protection and recovery, 
including air and space defense, police and army. In relation to energy infrastructures, 
the technology can help observe power networks from the air or ground, trace electric, 
gas or oil supply lines, sensing their states (and, if needed, directly accessing the 
disaster zones), also providing regular or emergent sentry duties at power 
installations, etc. 

Global and Battlespace Dominance. The DSL scenarios, using any electronic media, 
can self-spread, outline, and grasp distributed systems of different natures while 
establishing global dominance over them. They can analyze their internal 
infrastructures, finding strong and weak points, orient behavior, or destroy the 
infrastructures or the system as a whole if required. The approach, as an intelligent 
self-recovering super- virus, which is difficult to discover and kill by traditional 
means, can effectively employ advanced robotic facilities, like swarms of aerial and 
ground vehicles, to attack adversarial systems. 

7 Gestalt-Related 

Our approach may be considered as one of the first attempts to formalize and 
implement the notion of gestalt [11,12], under which the whole dominates over parts 
(being greater than the sum of them) with parts having sense only in the context of the 
whole, rather than vice versa. Gestalt theory represented the main departure from 
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atomistic vision of systems at the beginning of the last century. Many existing 
systems, especially distributed ones, are still based on the concept of predetermined 
parts (agents) that communicate with each other in an attempt to get the global 
behavior needed. The latter, with rapidly growing number of agents in complex 
systems (and starting from the agents level) is becoming more and more problematic. 

Within the approach offered, we have come to quite a different and higher level 
model of the system organization. Abstracting from system parts and their 
interactions, which may be emergent and varying at runtime, we can describe the 
needed global system behavior on a semantic level, where parts and their interactions 
may not be known in advance, and may dynamically appear (disappear too) just to 
maintain the global behavior needed. 

The technology developed allows us to automatically interpret global system 
scenarios in any networked systems (comprising internet hosts, laptops, mobile 
robots, mobile phones, smart sensors, and/or humans themselves). It allows us to get 
even higher — to describe what the system should do on the highest level, whereas its 
local and global behavior becomes a derivative of this description, which, in its turn, 
makes the system structures and operation as a. further derivative. 

8 Conclusions 

A novel ideology and technology, converting any distributed system into a universal 
spatial machine capable of solving complex problems on itself and on the surrounding 
environments, has been presented. This conversion can be achieved by implanting 
into the system sensitive points and its active doers, humans and robots including, of 
the same copy of a universal control module, communicating with other such modules 
via available channels. Their entire network, which may be dynamic and emergent, 
collectively interprets mission scenarios written in a special high-level language, 
which are defining system' s internal and external behavior. 

Created and modified on the fly, the scenarios can start from any component, 
covering the system at runtime through the cooperating interpreters. During the 
scenario evolution, any operations can be carried out throughout the distributed world, 
along with the needed movement of code, equipment and artificial or biological doers, 
humans including, as well as creation and maintenance of physical and virtual 
infrastructures supporting the missions. 

The approach offered can dramatically simplify application programming in 
distributed systems, especially robotized ones. As can be seen from the examples 
throughout this paper, programming multi-robot scenarios in distributed and dynamic 
environments in DSL may not be more difficult than, say, programming of routine 
data processing tasks in traditional languages like Fortran, C, or Java. 

The distributed robotized systems are of rapidly growing importance in many 
areas, and especially in defense, where robotic swarming on asymmetric battlefields is 
becoming a major dimension of the new military doctrine for the 21 st century [13]. 
The written above is much in line with these trends, allowing us to flexibly combine 
loose swarming with more classical command and control, which can help gradually 
transform fully manned into mixed and ultimately totally unmanned systems. 
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Other prospective applications of this work can be linked with economy, ecology 
and weather prediction — by using the whole networked world as a spatial 
supercomputer, self-optimizing its performance. 

The approach offered may also be compared with the invention of the first world 
computers and first high-level programming languages [14,15]. In our case, this 
computer may not only operate with data stored in a localized memory, but can cover, 
grasp, and manage any distributed systems, the whole world including, and can work 
not only with information but with physical matter or physical objects too. 
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Abstract. This paper presents a new Iterative Feedback Tuning (IFT)-based 
optimal state feedback control solution dedicated to a class of second-order 
servo systems with integral component. The state feedback controllers for these 
controlled plants are extended with an integrator to ensure the rejection of 
constant load type disturbances. Original IFT algorithms are suggested for the 
accepted state feedback controllers such that to set the step size in order to 
guarantee the convergence. An attractive convergence theorem based on the 
application of Popov's hypertability theory to the parameter update law in the 
IFT algorithms is offered. The theoretical results are validated by a case study 
concerning the position control of a DC servo system with backlash. 
Implementation issues are discussed and exemplified by real-time experimental 
results. 

Keywords: Implementation Issues, Iterative Feedback Tuning, Second-order 
Servo Systems, Real-time Experimental Results. 



1 Introduction 

The second-order servo systems with integral component play an important role as 
controlled plants in a large category of industrial applications including mechatronics, 
electrical drives, sub-systems in power plant control systems, positioning systems in 
manipulators, mobile robots, machine tools, flight guidance and control [l]-[8]. 
Those controlled plants are viewed as special cases of benchmark systems [9] — [1 1]. A 
convenient way to develop control solutions dedicated to these controlled plants 
makes use of linearized models at certain operating points. Therefore the parameter 
variation associated with the linearization is challenging when very good control 
system performance indices are required. The design and implementation involve 
more challenges when low-cost automation solutions are needed. 

J. A. Cetto et al. (Eds.): Informatics in Control Automation and Robotics, LNEE 85, pp. 99-|_ll_y 
springerlink.com © Springer- Verlag Berlin Heidelberg 20 1 1 
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The control solutions based on state feedback control systems are can cope with 
the accepted class of plants. The optimal state feedback control systems can fulfill the 
main control aims i.e. high performance indices in reference input tracking and 
regulation with respect to several types of load disturbance inputs. With this regard the 
improvement of the control system performance indices (e.g. settling time and 
overshoot) is enabled by the minimization of appropriately defined objective functions. 
The Iterative Feedback Tuning (IFT) [12], [13] is an alternative to the experiment-based 
minimization of the objective functions. The IFT algorithms employ the input-output 
data measured from the closed-loop system during its operation to calculate the 
estimates of the gradients and eventually Hessians of the objective functions. Several 
experiments are conducted per iteration and the updated controller parameters are 
calculated on the basis of the input-output data and the estimates. 

The extension of IFT according to [14] offers additional steps to improve the 
convergence properties of IFT while rejecting the disturbances. Several extensions of 
IFT to Multi Input-Multi Output (MIMO) systems are discussed in [15]— [18]. Linear 
applications to digitally simulated benchmarks are illustrated in [15], [16]. The need 
for faster gradient approximations and local convergence in IFT for multivariable 
processes are thoroughly discussed in [17]. Recently reported industrial applications 
of IFT include the control of chemical plants [18] and servo drives [19]. 

Many experiments are needed for more state feedback control systems and MIMO 
systems. The need to reduce the number of experiments per iteration has been 
highlighted in [14]-[17]. 

The IFT-based state feedback control meant for second-order servo systems with 
integral component has been suggested in [20]. Building upon [20] this paper presents 
twofold new contributions. First, original convergent IFT algorithms are suggested. 
They are based on the formulation of the parameter update law in the IFT algorithms 
as a nonlinear dynamical feedback MIMO system in the parameter space and iteration 
domain. Popov's hyperstability analysis results [21], [22] are applied in this context to 
derive a new stability theorem which guarantees the convergence of the IFT 
algorithms and gives an useful condition to set the step size. Second, a thorough 
discussion of an extensive set of real-time experimental results is done. 

This paper is organized as follows. The state feedback control system structure 
dedicated to the second-order servo systems with integral component is presented in 
Section 2. Section 3 focuses on the new IFT algorithm. Section 4 discusses 
implementation issues. A case study concentrated on the state feedback position 
control of a DC servo system with backlash is described in Section 5. The real-time 
experimental results validate the new control solution and IFT algorithm. Section 6 
highlights the conclusions. 

2 Control System Structure 

The second-order servo systems as controlled plants are characterized by the state- 
space model [20]. 
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where x = [x { =a x 2 = co] r is the state vector, a is the (angular) position, co is the 
(angular) speed, u is the control signal, y\ and y 2 are the controlled outputs, I 2 is the 
second-order identity matrix, and the superscript T indicates the matrix transposition. 
The two parameters in (1) are the process gain K ,K >0, and the small time 

constant or the sum of parasitic time constants T ,T > ■ 

The two transfer functions considering the input u and output <n, and the input u 
and output a are p (s) and p (j) , respectively, with the following expressions: 
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(l + sT s ) "' s(l + sT s ) 

The integral component can be observed in p (s) when the controlled output is 
y = x l = a. Such situations correspond to positioning systems. 

The state feedback control system structure is presented in Fig. 1. The dotted 
connection is used only when the experiments specific to IFT are conducted. That 
connection is not applied during the normal control system operation, therefore the 
control system structure is not a model reference adaptive one. 
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Fig. 1. IFT-based state feedback control system structure with integrator in the state feedback 
controller 



The main variables and blocks in Fig. 1 are: IFTA - the IFT algorithm, RM - the 
reference model, r - the reference input, e = r — y - the control error, k c = [K l K 2 ] 

- the state feedback gain matrix to be tuned by means of the IFT algorithm, 
P(s) = P (s) - the transfer function of the controlled plant with the controlled 
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output y = x l =a, y - the reference model (desired) output, e, = y d —y - the 

tracking error. 

The state-space model (1) can be reconsidered by including one additional state 
variable x 3 = x R which corresponds to the integrator inserted to the state feedback 

controller. Thus its gain K R will be subject to IFT as it is illustrated in Fig. 1. The 
extended state-space model of the process becomes then 
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where x ^ = \a co x„ Y = \x x„ Y is the extended state vector, I? is the second-order 

identity matrix, y = [y t y 2 y,Y is the controlled output vector, and the parameter 

K r is not included in the tuning scheme. K r is set prior to the application of IFT. One 
way to choose K r is to keep a connection between the steady-state value of r and the 
steady-state value of r x for which the desired r can be tracked by the steady-state 
value of y. That value of r x can be subject to the experimental identification of the 
state feedback control system. 

3 Convergent Iterative Feedback Tuning Algorithm for State 
Feedback Controllers 



The objective function / defined over the finite time horizon N is 



(4) 



where 



R" 



the parameter vector, which for 



K, 



P = [Pi = K x p 2 

The IFT algorithms [12]-[20] are applied to find the solution p* to the 
optimization problem 



p =argmin/(p) 

psXD 



(5) 



where several constraints can be imposed in relation with the controlled plant and the 
state feedback control system. One of these constraints concerns the stability of the 
system and SD stands for the stability domain with this regard [23], [24]. 

Solving the optimization problem (6) requires finding the parameter vectors that 

make the gradient _ equal to zero: 
3p 



a/ a/ 

3p 3p, 



dJ 

3p m 



(6) 
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Since the controlled output depends on p as y(p) and y d not, use is made of (4) and 
(6) becomes 

^i^bKp)-/] = o- (7) 

The partial derivatives fv should be calculated to obtain the components of the 

gradient, _ , i = 1 m ■ The experiments specific to IFT are conducted to obtain those 

dp, 
components. Use is made of the notation [20] 

a= *: <8) 

to highlight the partial derivative of the variable a with respect to the parameter p , 

i = \,m- 

To derive the IFT algorithms the following relations can be extracted from the state 
feedback control system structure presented in Fig. 1 : 

r x =K r r + K R x R , (9) 

u = K r r + K c x E , 

where the extended state feedback gain matrix K is 

K c =[-K, -K 2 K R ] = [-k c K R ] • (10) 

The matrix K in (10) highlights the parameter vector p which for m = 3 is 

p = [K, K 2 K R f . (11) 

Since y and u are functions of p the first and third equations in (9) yield 



y'=P«' 



u'=K c 'x E +K c x E ' 



(12) 



Next the second equation in (3) enables the following transformation of the second 
equation in (12): 

M '=K c 'y + K c y'. (13) 

The first term in the right-hand side of (13), K ,'y , needs to be added to the control 

signal to obtain the experimental scheme. That term contains the unmodified output 
vector (in the MIMO framework) obtained from the first experiment [13]. The second 
term in the right-hand side of (13), K y' , is measured from the state feedback control 
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system structure. Therefore the experimental scheme to calculate the gradients is 
presented in Fig. 2 where the blocks RM and IFTA (in Fig. 1) are omitted for 
simplicity. 
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Fig. 2. Experimental scheme to calculate the gradients in the IFT-based state feedback control 
system structure 



The first experiment specific to IFT, referred to also as the normal one, is 
conducted with the control system structure presented in Fig. 1 to measure the 
controlled output y. The next m = 3 experiments, i.e. the gradient experiments, are 
conducted with the experimental scheme presented in Fig. 2. These experiments are 
done separately for each parameter in K considering the zero values of the other 

m - 1 = 2 parameters because their derivatives with respect to the currently 
considered parameter are zero. 

The parameter vector must be updated after the experiments are finished. Newton's 
algorithm is generally used as a convenient technique which iteratively approaches a 
zero of a function without knowing its expression. The update law to calculate the 
next parameter vector p' +1 is 



■Y.Rr'^Ap')] 
3p 



(14) 



where i is the index of the current iteration / experiment, y > is the step size, 

cst\ (p')l * s me es ti mate of the gradient, and the regular matrix R ; can be the 

dp 
estimate of the (positive definite) Hessian matrix or the identity matrix. 

Popov's hyperstability analysis results will be applied as follows to the parameter 
update law (14) in order to derive a condition to guarantee the convergence of the 
IFTAs. First the domain of attraction of the update law as part of an IFTA is defined 
in terms of the following definition which is equivalent to that presented in [25]. 

Definition 1. Let p* be the global minimum of the function J ; R m — > R + defined in 

(4). The set II cz R m is called a domain of attraction of the update law (14) for the 
function J if 



lim, 



= p\ Vp°en 



(15) 



where p" is the initial parameter vector. 
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The convergence result is next expressed in terms of Theorem 1. 

Theorem 1. Let i { > be an arbitrary index of experiments / iterations and 
e n = const , e„ * • If the condition 



n 



'' r) J 

I(i 1 ) = Y j yj(est[—(p J )]) T (Rr i ) T pJ>-el Vi, >0 (16) 



is satisfied for the isolated global minimum p* then n is a domain of attraction of p* . 

Proof. The relationship (14) is expressed as follows as a dynamical feedback MIMO 
system in the parameter space and iteration domain in terms of the structure presented 
in Fig. 3, where the feedforward discrete-time linear time-invariant (LTI) block is 
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Fig. 3. Dynamical feedback MIMO system structure of (14) used in the convergence analysis 

p +1 =Ap +Bji\ 

▼'=Cp'+D(i', (17) 

A = B = C = I, D = 0, 

and the nonlinear (NL) feedback block is 

w'=y,R,"W^(v')]- (18) 

dv 

LTI is completely controllable and completely observable. The discrete transfer 
function matrix of the LTI, H(z) , obtains the expression 

H( z ) = C( Z I-A)- 1 B + D = (zI-ir 1 

= diag(l/(z-l), l/(z-l), ..., l/(z-l)), 

and it is a positive real discrete transfer function matrix. Since NL satisfies the Popov 
type inequality. 

/(J 1 ) = X( w 7 V>-e 2 , Vi^O . (20) 

which is equivalent to (16), the conditions for the hyperstability of the system (17), 
(18) are fulfilled [21], [22]. Thus the convergence of the IFTAs with the update law 
(14) is guaranteed. 

The new family of IFTAs dedicated to the considered class of stat feedback control 
system consists of the following steps. 
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Step 1. Conduct the normal experiment making use of the control system structure 
presented in Fig. 1, measure y(p')=x and calculate the reference model output 
vector y d . Next conduct the three gradient experiments in terms of the experimental 
scheme presented in Fig. 2 and measure the outputs that give the gradient of the 

controlled output — (n' ) • 
dp 

Step 2. Calculate the estimate of the gradient of the objective function 

est[— (p' )] = — V M- (p' )]' ly(p' ) - y" ] • (21) 

dp N ~ dp 

Step 3. Calculate the Popov sums I(i t ) in (16) and set the step size y. such that to 
fulfill the convergence condition (16) for any i v < j, < i and any e = const, e ^ 0- 

Step 4. Calculate the next set of parameters p' +1 according to the parameter update 
law (14). 

4 Implementation Issues 

Setting the reference model (Fig. 1) is important. This aspect should be related to the 
imposed control system performance indices and the accepted model of the controlled 
plant although it is achievable. Over ambitious reference models can results in the 
impossible fulfillment of the performance specifications. 

The first task of the state feedback controller is to ensure an initially stable control 
system. The pole placement design can be used with this regard in order to set the 
initial parameter vector p° . 

The identity matrix is recommended to play the role of the positive definite matrix 
R in (14) for low-cost implementations. Another alternative is to calculate the 

estimate of the Hessian matrix to play the role of R . . That calculation should be 

included in the step 3 of the IFTAs and an additional experiment can be employed 
with this regard. 

In many cases the actuator is characterized by a nonlinear input-output map caused 
by the actuator saturation. That is a problem because it introduces usually nonlinear 
behaviors in the plant's dynamics, thus it must be avoided. When making use of the 
integrator in the controller the importance of the actuator saturation is increased 
because actuators which enter deep saturation regions require usually longer time 
periods to re-enter the active regions of normal operation. 

The structure illustrated in Fig. 2 highlights that when the state vector is fed over 
the control signal it may cause the saturation effects. Therefore the experiments will 
provide estimates of the gradients which are different with respect to the correct ones. 

For the sake of simplicity an actuator with the active input range within -1 to +1 is 
considered as follows. One solution to solve the actuator saturation problem is to 
design the experiments such that the actuator does not enter saturation. Therefore the 
injected variable (in Fig. 2) must be in the active region of the actuator's input-output 



Convergent Iterative Feedback Tuning of State Feedback-Controlled Servo Systems 107 

static map. The injected variable can be scaled to its maximum value M , M > 0, of 
its measured dynamics. Generally speaking for an injected variable z t = 1, N , its 
scaled value added to the control signal is 

(zj, =z,/M, M = max I z, I • (22) 

k=l.N v ' 

Therefore it is guaranteed that the new variable to be injected, (z ) > is within the 

accepted domain of the actuator input. Some details concerning the way that the gradient 
experiments are influenced are offered in [20]. 

5 Case Study and Real-Time Experimental Results 

The validation of the new state feedback control solution and IFT algorithms is done 
in terms of a case study dedicated to the position control of a DC servo system with 
backlash. The experimental setup illustrated in Fig. 4 is built around the INTECO DC 
motor laboratory equipment. It makes use of an optical encoder for the angle 
measurement and a tacho-generator for the measurement of the angular speed. The 
tacho-generator measurements are very noisy. The speed can also be observed from 
the angle measurements. The control system performance indices such as settling time 
and overshoot can be assessed easily. 

It is accepted that y = x, = a in relation with (1) and (3) and Fig. 1. The plant is 

characterized by the parameters K =139.88 and T = 0.9198s, obtained in terms of 
experimental identification. Part of the real-time experiments is presented here and it 
makes of the initial parameter vector set to set to p° =[0.0132 0.0126 0.005] r in 
order to stabilize the control system. 

A r = 150rad step type modification of the reference input was applied. Therefore the 
parameter K. was tuned at the value K =0.0133 and dropped out of the objective 
function (4). That value of K. was obtained by steady-state calculations to ensure a 

desired gain between r and y. The sampling period was set to 0.01 s. The continuous-time 
RM is characterized by the transfer function 

G RM (s) = l/(s 2 +l.5s + l) (23) 

which is next discretized. 




Fig. 4. Experimental setup 
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The real-time experimental results that illustrate the behavior of the control system 
in terms of the evolution of the controlled output before the application of the IFT 
algorithm are presented in Fig. 5. 
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Fig. 5. yr (dotted line) and y (continuous line) versus time considered before IFT 



The IFTA presented in Section 3 was applied. The parameters in the IFTA were set 
to y. =0.0001 and R = 1 , The evolution of the controlled output after 12 iterations 

is presented in Fig. 6. 
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Fig. 6. y d (dotted line) and y (continuous line) versus time considered after IFT 



The control system performance enhancement due to IFT is highlighted. It is 
reflected by smaller overshoot and settling time. The control system performance 
enhancement is also pointed out in the evolutions of the control signal and speed 
x, = CO illustrated in Fig. 7 and Fig. 8, respectively. The oscillations in the control 
signal and speed are caused by the backlash. 
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Fig. 8. x 2 versus time considered before and after IFT 
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6 Conclusions 



This paper has presented a new approach to the convergent IFT-based design of a 
class of state feedback control systems meant for a class of second-order systems with 
integral component. The convergent IFT algorithm proposed here is sufficiently 
general to be applied without additional difficulties to the state feedback control of 
systems of arbitrary order. 

The real-time experimental results in the case study validate the optimal state 
feedback control solution offered in this paper. It shows the power of IFT because the 
IFT-based design of the control system offers better performance indices compared to 
the situation prior to the application of the IFT algorithm. The performance indices 
are very good for the nonlinear controlled plant although the theoretical approach is 
based on the linear or linearized mathematical model of the controlled plant. 
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The future research will compensate for the limitations of the theoretical approach 
presented here. They involve the initial controller tuning and the guarantee of 
obtaining of the global optimum after a relatively small number of iterations. Quasi- 
optimal state feedback control systems can be designed currently. Emphasis will be 
given to the transfer of the results from the linear case to the fuzzy one and to the 
extension to other classes of controlled plants [26]-[32]. 
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Abstract. This paper deals with the design of a linear quadratic (LQ) control 
law combined with a multimodel approach for variable-speed, variable-pitch 
wind turbines in order to improve their performances, especially the system 
global dynamic. The control law is based on a linearization of the system 
around a selected set of operating points. Hence, a set of linear local models 
(sub-models) is defined to describe the wind turbine functioning in the high 
wind speeds operating range. A state observer is used to estimate the 
immeasurable state variables. Thereafter, a global asymptotic stability analysis 
is developed by solving a bilinear matrix inequality (BMI) feasibility problem 
based on the local stability of the sub-models. 

Keywords: LQ Controller, Multimodel Approach, Global Stability, LMI, Ly 
Apunov Equations. 



1 Introduction 

Nowadays, the growth of the utilization of the wind turbines is more and more 
important since they are producing carbon-emission-free electricity. Until today, only 
classic control laws, such as P, PI or PID controllers, are used in the wind turbines. 
However, the performance of these controllers is limited by the high nonlinear 
characteristics of the wind turbine and by the appearance of new control objectives 
required by the grid-codes; the reason why advanced control research area is 
improving every day. 

In the first axis of this paper, an LQ controller, which had been advocated by many 
researchers, is designed with a multimodel approach, for pitch regulated variable speed 
wind turbines operating at high wind speeds, in order to guarantee an optimal behavior 
for the studied process. However, this technique still presents some limits to satisfy all 
the control objectives especially those concerning the system global dynamic. This paper 
aims then to present an issue for this problem by adding an exponential term in the 
quadratic cost function. 



*This work was originally presented in the ICINCO'09 Conference, Milan, 2009. 
This work was supported by the CMCU project number: 08G1120. 
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The second section deals with the asymptotic stability analysis of the global 
sys-tem by solving a set of BMI according to the Lyapunov theorems. In fact, the 
stability study is necessary and important to illustrate the effectiveness of the 
presented strategy. 

Finally, the simulation results realized on Matlab Simulink are presented and 
discussed. 



2 Wind Turbine Modelling 

2.1 Wind Turbine Description 

The considered wind turbine (Figure 1) is modeled as two inertias (the generator and 
the turbine inertias respectively J g and J T ) linked to a flexible shaft with a mechanical 
coupling damping coefficient d and a mechanical coupling stiffness coefficient k. This 
model is widely used in the literature [1]; [2]. 







Fig. 1. Wind turbine dynamic model 



where Q T and Q g are the turbine and the generator rotational speeds, T em is the 
generator torque, T mec is the drive train mechanical torque and T aero is the torque 
caught by the wind turbine which is expressed by: 



pKR 5 a 2 T 

I 3 



■c„a,/3) 



(i) 



where p is the air density and R is the turbine radius. 

The power coefficient c p (Figure 2) is a non linear function of the blade pitch /? and 
the tip speed ratio X depending on the wind speed value v and given by: 



1= &T- R 



(2) 
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Fig. 2. Power coefficient curves 



The dynamic response of the rotor is given by: 
J T h T =T -T 

T T aero m< 



(3) 



The generator is driven by the mechanical torque and braked by the electromagnetic 
torque. Reported to the low speed shaft, the characteristic equation is the following: 



J g-ls-^g -Is ~ * mec ^-1 er, 

where G is the gearbox gain and: 



(4) 



Q 



g —Is 



(5) 



J , = G 2 J 

g-ls g 



And the low speed shaft torque T mec results from the torsion and friction effects due to 
the difference between the generator and the rotor speeds [3], It's defined by the 
following equation reported to the low speed shaft: 



f mec =k.(n T -n g _ ls )+d.(Ci T -Ci g _ ls ) 

The pitch actuator dynamic is described by a first order system: 



(6) 



(7) 



Pref represents the control value of the blade-pitch angle /? and Tg is the time constant 
of the pitch actuator. 
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2.2 Linearization and State Representation 

The wind turbine is a complex non linear system presenting several difficulties in 
study and control. It seems then more suitable to describe it with a set of linear local 
models valid in different operating points corresponding to different levels of wind 
speed values. The principle of this method is used in several techniques. In this paper, 
we use the multimodel approach which was the subject of many research works [4]. 

For the studied system, we define a multimodel base made of four local models. 
The equivalent instantaneous model, as described in Figure 3, is obtained by a fusion 
of only two valid successive models. The choice of these models depends on the wind 
speed value. 

The weighting coefficient /v, is the validity value of the model M, and it can be 
expressed by: 



tt=l- 



(8) 



Multimodel base 



1 


Model M 1 








Model M 2 




• 
• 
• 


\ 


Model M N 


V 





\ 



Valid local models 



< ^r^\ 



Model Mi 



Model M 



i+i 



Equivalent model 



\<^> 



M=iAi.M,+ n hl . 
M M 



Fig. 3. Wind-turbine multimodel description 

r, is a normalized residue measuring the error between the instantaneous and the valid 
local model wind speed values (respectively v and v,). When M, and M i+1 are the valid 
models, the residue can be expressed as: 



V, 



v.. + V, 



+1 



(9) 



Thus, the validities satisfy the convex sum, such that: fij + // (+1 — 1 . 

To obtain the local models, the system should be then linearized around the 
operating point. The non-linearity of the system is due to the c p characteristic which is 
used in the expression of the aerodynamic torque. We need then to linearize the 
expression (1) of T aero around an operating point (o.p) defined by the wind speed 
value \>{[ [1]; [7], We can define: 
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AT = 



dT 



where: 



XL 



AQ T + 



dT 



dj8 



Afi = a r AQ T +b r A(3 



1 3 v , 
a. =-.p.n.R ' . — ' 

2 Q 



dcU,j3) c p 



T _ nam 



1 , V 

b=-.p.n.R\ — ! , 
'2 Q r 3^ 

1 /Wffl / 



3/1 A 

3c (A,/?) 



(10) 



(11) 



and: 



9 Q T 

(q n \ ' T _nom' aero _nom 

i nom ' Hi nom I m 3 

- ; p.7t.R 2 y. 



(12) 



The symbol A represents the deviation from the chosen operating point corresponding 

lO \zdT_norm ^ g-ls-noim Pi-norm -* mec-notm -* em-nom anU r nam) Wnere. 1 em-nom and 1 m ec-nom are 

respectively the nominal values of the electromagnetic and the mechanical torques. 

Thereafter, the linearization of the non-linear system expressed in equations (3), 
(4), (6) and (7) around an operating point gives a state space representation of the 
form below: 

\x = A.x+B i u 
y = q.x+D ( M (13) 

where x, u and y are respectively the state, control and output vectors defined as: 



x = 



" AQ T ~ 






A/? 


>y = 


~AQ T ~ 
AP 


AT 

mec 







and u = 



AT 



(14) 



Notice that P=T em .Q. g designates the generated electrical power. This leads to write 
around an operating point: 

(15) 

ap = g.(t An , +at n , ) 

\ em_nom g—ls em g-is _nom J 

Hence, A,, B h C, and D h which are respectively the state, input, output and 
feedthrough matrices, are defined as follows: 
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A = 



a. 





b 


1 


J, 


J, 


J T 











1 



— 



a. .d db. 

k + — — —k — '- —d. 

J, J, 



C 







G.T 








g-ls 



f 1 1 ' 

— H 

\ J T J g-ls J/ 



,B = 







and D 



,0 G.Q , 



g-ls 


d.G 



J 



-Is J 



(16) 



3 Controller Design 

The control task is based on the objective of regulating the rotor rotational speed and 
the generated power by acting on two control variables: the electromagnetic torque 
T em and the regulating pitch angle p ref . 

The LQ control strategy had been advocated by many research works [3]; [8]; [9]; 
[10]; [11]. This technique presents a good compromise between the performances 
optimization and the minimization of the control signals by the use of a quadratic cost 
function. However, it also presents the disadvantage of the non possibility of 
controlling the global system dynamic. In this paper, a solution that can partially solve 
this problem is presented. 

This controller aims to minimize the following quadratic criterion J : 



1 I 



J = -\(y T .Q.y + u T .R.u).e 2a 'dt 



(17) 



where Q and R are diagonal positive definite matrices. 

The term y T -Q.y expresses the performances optimization, the term u T .R.u 
expresses the minimization of the control signals and the term e 2at allows the 
performances improvement of the classic quadratic criterion. It leads to the placement 
of the system poles on the left of -a. The criterion can be rewritten as follows with an 
input-state cross term: 



1 + . 



J = — j (x T .Q l .x+2.x T .N.u + u T .R l .u).e 2a 'dt 



(18) 



where Qj, Rj and ./V are defined as: 
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Q = C T .Q.C 

R X =R + D T .Q.D 
N = C T .Q.D 



(19) 



For this criterion, the optimal gain can be calculated from the following Riccati 
equation: 



V '.L + L.A ia -(L.B i + N l ).R?.(B i T .L+NT) + Q 1 = 

A ia =A i +aJ 

K i =R-\(B i T .L+N 1 T ) 



(20) 



where / is the identity matrix. 

Since the dynamic of the pitch actuator should not be changed, the controller is 
designed in two steps. In the first step, we consider the blade pitch angle /? and the 
electromagnetic torque T em as control variables instead of fi re f and T em . The state 
representation becomes then: 



K =A il X r+ B il.Ul 

y = C il x r + D il u 1 



(21) 



where: 



AQ. 



Ml 



g-ls 



AT n 



AQ r 

AP 



and u 



1 



A/3 

AT„ m 



(22) 



And: 



Hi 



c. 



k + 



i 



a-.d 



-k —d. 





J T 

1 


J g-ls 

1 

— + — 



1 



11 



il 



o G.r 



em—nom 



K J T J 'g-ls J J 

10 
and D- , = 

ll \0 1 



h 



J 



d.b- 



g-ls 
d.G 



J 



(23) 



g-k J 
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From this representation, the optimal gain K 



K 



K 



is calculated such that: 



'; 



Mi 

AT 



-K n .x r 



(24) 



The relation (24) leads to the following optimal control law using the global state 
vector as shown in Figure 4: 



u = —K- .x 



K: 



K 



i-pref 



K 



i Tern 



(25) 



with: 








' K =(K 


^ +T , 




K = K 

i _ Tern il _T 


m .T 


■ 


k_, B a _ T 


,] = * 






1 


o o" 




T = 


1 












1 


> 


Ren 


mrk: 





(*,._, + *r K n_f A n)- T > + *,*n_,- B n_,Z -^n.pA.^.^n 



(26) 



and T = [0 1 0] 



We notice that the mechanical torque is an immeasurable variable. It seems necessary 
then to use a multimodel state observer to estimate its value [8]. 
We can write: 









AQy 


X = 


w = 


AD , 
g-ls 

v W J 




x 4 = AT mec 


f w V- 


A ll A 12 


A x 4 




_ 


A 21 ^22. 



(27) 



B^ 



The estimated value X4 = ATmec of JC 4 = AT can be designed in the following 

form: 

r 
z = A, , .w + Ajj ,z + B^ .m 

Xa = z + L.w 



(28) 
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The following relations are used in order to calculate the observer matrices making 
the observing error £ = X 4 — X 4 converge into zero: 



*21 " hi + h.2 



L-L.A 



11 



A 22 ~ ^22 L ' A \2 
B 2 = B 2 ~ LB 1 



(29) 



The observing gain L is chosen by pole placement in order to give the observer a 
faster dynamic than the system's one. 

Then, around a given operating point, the control law is calculated as given in (30): 



-K- .x 

AQ_ ,. (30) 



■Is 

ATmec 



wind 
(v f <v<v, +; ) 



g—ls—nom 




mec-nom 



4 Stability Study 



Fig. 4. The LQ controller design 



The quantum advance in stability theory that allowed one the analysis of arbitrary 
differential equations is due to Lyapunov, who introduced the basic idea and the 



122 



N. Khezami, X. Guillaud, and N.B. Braiek 



definitions of stability that are in use today. The concept of Lyapunov stability plays 
an important role in control and system theory. 

In the following study, the multimodel state observer is not taken into 
consideration. 

As we define a global model M by fusion of two successive local models M, and 
M i+h the characteristic matrices of the system (13) can be obtained by: 



A i ~ ^k A k +M k+l A k+l 



B i =^k- B k + ^k+V B k+l 

C i = M k £ k + M k+1 - C k+\ 
D i = Mk .D k+Mk+r D k+l 

The input vector is calculated by: 

u =-W- K k + V k+ Y K k+ l)- 
Hereafter, the state vector can be represented as: 

x = lXM i .Mj.(A i -B i .K j ). X 






G:: + G 



J' 



(31) 



(32) 



(33) 



where: G- • = A- - B- .K ■ 

U i i J 

To study the global asymptotic stability of the above system supplied by the 
multimodel LQ control, the first necessity is to analyze the stability of every local 
model. As we focus here especially on the closed-loop system, the criterion of 
stabilization consists then in finding, for a local model M„ a positive definite matrix 
P that satisfies the following LMI [Chedli, 2002; Liberzon and Morse, 1999]: 



G U .P + P.G U <0 



(34) 



In the case of the multimodel systems, an extra condition is to add to the LMI (34) in 
order to guarantee the global stability [12]; [13]; [14] and it consists in: 



$.P + P.Q ij <0, i<j 



(35) 



G, +G. 



where Q = 

2 

And in our case, only two successive local models are valid at a time, which means 

that this condition will be considered for i=l to 3 zn&j=i+l. 
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5 Simulation Results 

The proposed control approach and the stability analysis of the controlled system 
have been illustrated through simulations on Matlab Simulink. 
The simulated wind turbine parameters are presented in Table 1 . 

Table 1. Wind turbine parameter values 



Parameters 


Values 


Air density 


P 


1,22 Kg/m 


Turbine radius 


R 


40m 


Nominal power 


p 

1 nam 


2MW 


Nominal speed 


*^T-nom 


18 rpm 


Optimal power coefficient 


^■p-opt 


0.4775 


Optimal speed ratio 


"■opt 


9 


Gearbox gain 


G 


92.6 


Turbine inertia 


JT 


4.9x10" N.m.s 2 


Generator inertia 


Jg 


0.9x10" N.m.s 2 


Mechanical coupling damping coefficient 


d 


3.5x10 s N.m'.s 


Mechanical coupling stiffness coefficient 


k 


114x10" N.m 1 



Table 2 describes the four local models multimodel base used for the simulations. 
From this base, four optimal gains are calculated. 

To calculate the linearization coefficients a, and £>,, the following c p empiric 
expression relative to 2MW wind turbines is used: 



c =0.18x 

p 



90 



0.4+0.5/1 



-6.8-0.115y# 2 



xe" 



(36) 



Table 2. Multimodel base parameters 



Local model 
Mi 


Wind speed 
vi (m/s) 


Pitch angle 

Pi-nom f / 


Ml 


11.6 


1.1 


M2 


14 


8 


M3 


17 


11.1 


M4 


25 


15.4 



And thus, the stability feasibility problem consists in solving 8 LMI as shown after: 



P>0 (I LMI) 

Gji P + PG ii<^ i 
Qj j .P + P.Q ij <0,i 



1..4 (4 LMI) 
l.3,j=i + l (3 LMI) 



(37) 
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The simulation leads to the following result: 

' 8.848 -8.335 -0.008 0.298 

-8.335 8.101 0.007 -0.258 

-0.53 0.523 0.026 -0.001 

0.298 -0.258 -0.001 0.084 



(38) 



Finding this positive definite matrix P is a sufficient condition proving the global 
stability of the control technique presented above. 

For the simulations, we had chosen to place the closed loop poles for the local 
models at the left of -a=-0.5. This gives the following poles for each local model: 



1" local model: 



2 nd local model: 



3 rd local model: 



4"' local model: 



-1.032 + 12.24/ 

-1 .032-12.24/ 

-1 

-1.069 



-1.032 + 12.24/ 

-1.032-1224/ 

-1 

-1.06 



-1.033 + 12.24; 

-1.033- 12.24/ 

-1 

-1.049 



-1.037 + 12.24/ 

-1.037-12.24/ 

-1 

-1.046 



Thus, we can see that the pitch system pole (-1) is invariant for the four local models, 
and that all the other poles have their real parts less than -a. 

To test the performance of this control strategy, a series of simulation for several 
wind steps has been performed to show the improvement of the studied controller 
against a classic multimodel LQ controller [8]. 

The local models poles for the classic LQ strategy have the following values: 



1" local model: 



2"' local model: 



3" local model: 



4" local model: 



-0.346 + 12.25/ 


-0.346- 


- 12.25/ 


-1.162 + 0.868/ 


-1.162- 


- 0.868/ 



p 5 = 



-1.2 + 12401/ 


-1.2- 


12.401; 


-2.693 


+ 2.013/ 


-2.693 


- 2.0128; 




-2.148 + 12.845; 
-2.148-12.845; 
^U 08+2.418; 
-4.1081 -Z418i 






15 20 25 30 35 




Fig. 5. Comparison simulation between classic and improved LQ control laws 



The Figure 5 presents a comparison simulation between the two control laws. For this 
simulation, only the turbine rotational speed response for a wind step of 0.5 m/s is 
presented for the four local models. 
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In the comparison between both strategies, the system response for the proposed 
controller has indeed kept almost the same response time (about 4s), unlike the case 
with the classic LQ where the response times are varying from 12s for the 1 st local 
model to 2s for the 4 th local model. 

Compared to the proposed strategy, the classic multimodel LQ control law shows 
responses with a more oscillating transient mode. 

The improvements of the multimodel LQ controller consist in a more damped 
oscillatory mode and a faster dynamic than the classical control mode with an almost 
fixed response time for all the local models. 

The simulation of the system with the proposed control strategy for a variable 

wind speed between 12m/s and 25m/s leads to the results presented in the curves of 
Figure 6. 

The Figure 6 (a) illustrates a realistic aspect of the wind speed as described in a 
method elaborated by C. Nichita in [15]. From this aspect, the controller allowed a 
good regulation of the generated electrical power (Figure 6 (d)) and the rotation 
speeds of both the rotor (Figure 6 (b)) and the generator (Figure 6 (c)) around their 
rated values with taking into account the fatigue damage since the mechanical torque 
(Figure 6 (e)) maintains an almost constant value which thereby leads to have 
alleviated mechanical loads. As shown in Figure 6 (e), the mechanical torque is nearly 
equal to the estimated value. 

The variations of the control signals: the electromagnetic torque presented in 
Figure 6 (g) and the control pitch angle in Figure 6 (h) are smooth and have 
acceptable variations. 




10 20 30 40 50 00 70 80 90 100 



10 20 30 40 50 00 70 SO 90 100 



iNvWwAWV^^vVn^Vwv' ! -a/i 



10 20 30 40 50 00 70 80 90 100 
x 10* (9) 



10 20 30 40 50 00 70 80 90 100 




10 20 30 40 50 60 70 80 



"-a/" 



Fig. 6. Variation of the system variables 
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6 Conclusions 

This paper dealt with a technique of designing a multimodel LQ regulator allowing to 
partially control the process global dynamic, and with a study of the global 
asymptotic stability of the controller by means of a set of LMI. The proposed strategy 
presented a compromise between different control objectives: optimizing the 
performances of the different system variables especially generating an electrical 
power of a good quality, minimizing the control efforts, alleviating the drive train 
dynamic loads and controlling the global dynamic of the studied process. The 
simulations results showed good performances of the controller with acceptable 
mechanical stress. But, satisfying such a trade-off between all these objectives is 
indeed difficult and the cost is however some high forces on the pitch actuator. These 
effects brought more challenges in the system analysis to improve the obtained results 
in order to control actively the system dynamic and to totally damp the oscillatory 
mode. 
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Abstract. Since its introduction in the early 1980's, CAN has become the 
de-facto communications protocol employed in vehicle and industrial control ap- 
plications. Before any new device can claim to support CAN-connectivity, exten- 
sive conformance testing is normally required to demonstrate compliance with 
the protocol at the physical and data link layers. To help standardize the nature 
of the specific test plans and documentation required for this testing, the ISO has 
published a set of draft standards specifically for CAN conformance testing. To 
date, most commercial CAN controllers and transceivers have been implemented 
at the silicon level, either in the form of dedicated IC's or as on-chip peripherals 
of embedded devices. The practical implementation of CAN conformance testers 
has been realised using dedicated hardware and specially written analysis soft- 
ware; this is a practical approach when testing and verifying conformance prior 
to high-volume IC manufacture. However, recent years have seen an increased in- 
terest in the employment of programmable logic devices such as FPGA's for the 
implementation of CAN controllers and CAN-enabled devices. Such 'soft core' 
implementations are often realised in small-volume (or sometimes even one-off) 
batches; in such circumstances, cost and availability reasons may dictate that de- 
velopers simply cannot employ traditional CAN-conformance testing equipment. 
To help alleviate this problem, this paper proposes a low-cost and easily imple- 
mented method which will allow developers to fully test a CAN soft core im- 
plementation. The method allows developers to verify a CAN core against the 
relevant ISO standards using only (low-cost) off-the-shelf development boards, 
coupled with a simple analysis tool such as Chipscope. Finally, the paper exten- 
sively describes the use of the test bed in the verification of an open-source CAN 
soft core implementation. 

Keywords: Controller area network, Conformance testing, Soft core, Network 
protocol verification. 

1 Introduction 

Conformance testing is an integral part of the development stage of any network pro- 
tocol implementation. When performed correctly conformance testing verifies, to an 
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acceptable degree of confidence, that the implementation of given set of protocol spec- 
ifications has been correctly interpreted by the designers and has been instantiated in a 
form that is free from errors. 

Since its introduction in the early 1980's, the Controller Area Network (CAN) pro- 
tocol has become the de-facto communications protocol employed in vehicle and in- 
dustrial control applications. In light of the popularity of CAN, the ISO has developed 
a standard exclusively aimed at CAN conformance testing. Before any new equipment 
design can claim to be CAN conformant [ 1 1 , evidence is required that shows that the 
testing procedures outlined in [2] standard have been performed and passed without 
problem. The ISO document not only specifies different types of tests that must be per- 
formed for conformance testing, but also specifies a Test Plan (TP) architecture based 
on the [3|. The required TP is shown in figure[T] As can be seen from this figure, the TP 
architecture indicates that the tester should be divided into two parts. The first compo- 
nent is the Lower Tester (LT) which provides the test pattern generation and analysis. 
The second is termed the Upper Tester (UT), which is required to contain the software 
to control the CAN Implementation Under Test (IUT). The UT is normally a host pro- 
cessor or programmable device of some kind, and also provides coordination to conduct 
the tests between the LT and the IUT H. The UT receives stimulus (with details of the 
test being performed) from the LT, and generates messages passed on to the IUT. The 
IUT then processes these messages, and both the UT and LT components monitor its 
behaviour for consistency with the CAN protocol. If the result is satisfactory, the test 
is considered passed and testing proceeds to the next conformance test. It should be 
noted that the testing procedures that are required to be implemented include coverage 
of common error conditions, randomized tests and also bit timing tests. Most tests are 
critical, and the latter category bit timing contains a number of tests that can be diffi- 
cult to localize, and a suitable means is required to capture and display multiple logic 
signals over an appropriate time scale. This typically requires the use of dedicated hard- 
ware and Logic analysers (5). 

Traditionally, CAN controllers and transceivers have been implemented at the sil- 
icon level, either by dedicated IC's or as on-chip peripherals of embedded devices. 
Practically, the implementation of CAN conformance testers has been done using ded- 
icated hardware and specially written analysis software, which is a practical approach 
when testing and verifying conformance prior to high-volume IC manufacture. The mo- 
tivation for the current work is as follows. Recent years have seen an increased interest 
in the employment of CAN-enabled devices implemented by programmable hardware 
devices such as FPGA's. By their very nature, such 'soft core' implementations are 
often needed in small-volume (or even one-off) batches. In these circumstances, cost 
and availability reasons often dictate that it is not practical for developers to use tradi- 
tional CAN-conformance testing equipment. An ongoing project within the Embedded 
Systems Laboratory (ESL) at the University of Leicester requires the use of such a 
CAN soft core [6|. To help alleviate this problem, this paper proposes a low-cost and 
easily implemented method which will allow developers to test a CAN soft core imple- 
mentation for conformance to the relevant standard without the need for expensive or 
proprietary hardware interfaces and logic analysers. 
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Fig. 1. ISO 9646-1 Test Plan Architecture 

The remainder of the paper is organized as follows: In the next Section, we will 
describe the Controller Area Network in brief and then will present a review of dif- 
ferent CAN conformance testing implementations of the before and after ISO standard 
evolved. Section 3 describes the formation of the current test bed for CAN conformance 
testing. Section 4 presents the case studies involving two of the tests being carried out 
by the proposed approach to CAN conformance. Section 5 will present the analyti- 
cal comparison of our approach to the other techniques used for conformance testing. 
Section 6 presents our initial conclusions. 

2 Controller Area Network 
2.1 Protocol 



The Controller Area Network (or simply CAN) is one of the most widely employed 
protocols for creating distributed embedded systems, with applications as far ranging 
as vehicle electronics, process control and many other important industrial applica- 
tions [1|. Some of the key features of CAN that have led to its widespread use in- 
clude low overheads, non-destructive bitwise message arbitration, low message latency 
and good error detecting abilities; all features which are required for control applica- 
tions running on embedded processors 17181 . As mentioned, the protocol employs a 
unique non-destructive priority-based arbitration scheme; when multiple nodes attempt 
to transmit messages simultaneously this mechanism ensures that the highest priority 
message gains first access to the bus. If priorities are carefully assigned to the messages, 
and appropriate timing analysis is performed, CAN may be used to implement several 
different types of time-critical systems (e.g. see [7I8I9ID . 
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The wired- AND nature of the physical layer, which is used to achieve the aforemen- 
tioned priority-driven arbitration, requires that all nodes in the network achieve a logical 
consensus on the instantaneous bit-patterns appearing on the bus lines. This particular 
requirement of the protocol acts to severely limit both the maximum transmission speed 
and bus length of a given CAN network; the maximum transmission rate is inversely 
proportional to the length of the bus, and has an upper limit of 1 MBit/s; due to its design 
a CAN frame may carry up to a maximum of 8 data bytes. In addition, this mechanism 
places extremely specific requirements on the nature of the conformance testing that 
must place for a CAN enabled device; a significant proportion of the ISO test standard 
is devoted to this single, critical aspect of the protocol. 

2.2 CAN Conformance Testing 

One of the earliest CAN prototype controllers was named DBCAN iflOll . This imple- 
mentation was tested using a logic analyser and a pattern generator circuit. As there was 
no standard for conformance testing at the time the prototype was developed, a commer- 
cial basic (as opposed to full) CAN controller was used as benchmark for verification. 
A major disadvantage of this scheme was the use of external interface modules to visu- 
alize the state of different DBCAN registers, and the testing procedure was somewhat 
limited in the number of signal channels that could be simultaneously analysed. Since 
this is a needed requirement in the case of ISO standard conformance testing the ability 
to visualize the state of large numbers of CAN registers simultaneously is a prerequisite 
such a setup is limited in this respect. 

A Hardware emulation technique was used to verify a CAN soft core in 1111 : firstly, 
the synthesized net list is downloaded into a hardware emulator. This emulator is config- 
ured by a PC and the communication between the two is carried via a specially designed 
interface card connected to the EISA bus; this emulator is also connected to 2 commer- 
cially available CAN chips. The drawback with this technique is that again, customized 
hardware along with software especially written to carry out the conformance testing 
is required. Additionally, to emulate the bus failures and potential error conditions on 
the bus a manual technique of connecting CAN bus to the output of individual nodes 
is employed, which lacks efficiency and is not robust enough to cover all the scenarios 
given within ISO DIS 16845. 

A slightly different verification technique was reported by lfl2l . Their technique 
employed custom design boards with 8051 microcontrollers and SJA1000 CAN con- 
trollers, but this method involved the design of specialized interface hardware and 
boards to assist with the testing plan. Specialised verification architecture for testing 
automotive protocols (including CAN) at both the module and chip level was proposed 
by [fT~3l . Again, this work requires a specially designed CAN verification component 
as part of the silicon, while the selection and implementation of actual test sequences, 
along with the selection of a suitable means to monitoring bus signals, is left open for 
the tester. 

With respect to soft core CAN implementations, the CAN e- Verification (CANeVC) 
test bench has previously been described 0141 . This commercial test facility requires 
a CAN specification core to be embedded in the netlist; this core then runs specific 
tests to verify the behaviour of the CAN soft core. Again, this technique involves a 
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time consuming development of a test bench using an expensive commercially avail- 
able verification IP ; additionally, compatibility issues often arise when using CAN 
implementations other than the proprietary implementation and only a limited number 
of programmable logic devices are supported. 

Finally, several experimental implementations (such as that reported by ITT31 to mea- 
sure single parameters - such as CAN bit errors - rather than perform complete con- 
formance testing have been described in the literature. Such implementations have typi- 
cally used complex and non-trivial means, requiring customized hardware and software. 
In summary then, it can be observed that - to date specialised hardware and / or soft- 
ware has been required to assist with CAN testing plans. In the following Section, a 
novel testing approach that relies only upon the use of low-cost, standard off-the-shelf 
hardware and software is described. 

2.3 Protocol Limitation 

However, the CAN protocol itself is not without its drawbacks; although the basic raw 
protocol is suitable for use in many soft real-time systems, it suffers from several sig- 
nificant problems with respect to hard real-time systems, for example in safety-critical 
distributed brake-by-wire systems. These drawbacks include redundancy issues, atomic 
broadcast problems, lack of protection from babbling idiot failures and information 
throughput restrictions. Although research has shown that many of these issues can 
be dealt with by the creation of higher-level, software-based protocol extensions (e.g. 
see 17181161 ). some proposed solutions are either complex to implement placing a sig- 
nificant computational overhead on the host CPU or simply cannot be implemented in 
software and a hardware solution must be adopted [ 17 1 . Although the proposed changes 
are conceptually quite straightforward, implementing them directly in silicon is costly 
and has proved to be problematic for small-volume requirements. A workable solution 
to this problem is to implement the protocol on a PLD; the required modifications may 
then be achieved with relative ease. This solution brings with it another, related prob- 
lem; before verification of any modifications can take place, it must be shown that the 
basic soft core CAN implementation is fully conformant to the protocol, a potentially 
costly and time-consuming procedure in its own right. 

To help alleviate this problem, the current authors have previously proposed a low- 
cost and easily implemented method in a technical report [18]. This report essentially 
describe the application of the techniques described in the previous Section to the con- 
formance of a CAN soft core. The following Section contains a description of the test 
facility that was employed in these studies. 

3 Test Bed 

Real-time testing of a CAN implementation is quite a complicated procedure, and in 
this case for practical reasons, no specialized hardware and software was available to 
generate the required testing patterns and monitor the behaviour of the CAN soft core. 
For this reason, it was decided to use only low-cost off the shelf components. 

In addition to these standard hardware parts, the Chipscope analysis tool lfT9l was 
used to visualize and capture the behaviour of the soft core, allowing verification of the 
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Fig. 2. Conformance Test Bed 

testing results. Chipscope is a Xilinx testing tool which is implemented by inserting 
a small core onto the device to be monitored, allowing multiple signal channels to be 
captured via a JTAG interface. Up to 16 internal signal ports can be analysed in a single 
core, and each port can have up to 256 signals. Multiple cores can be attached in a FPGA 
to increase the number of signals [20|. In comparison to other means for capturing 
multiple FPGA signals, Chipscope retains the key features required but is a fraction 
of the cost. Additionally, to support one-off conformance testing plans without causing 
excessive costs, a fully-featured evaluation version is available for a 60 day period a full 
testing plan can be performed in such a time frame. Hence these features of Chipscope 
made it an obvious choice for our CAN conformance test bed. The new test facility 
is shown schematically in figure [2] In the next paragraphs we give a full list of the 
hardware and software components and tools used in building the Test Bed. 

3.1 Hardware 



Two Integrated boards with FPGA's (programmed with CAN soft core) and an 
ARM 7 working as a Host controller. These boards are named as SCI and SC2. 
The purpose of using two soft cores is to verify simultaneous behaviour as a CAN 
Transmitter/Receiver as well as to generate special patterns on the CAN bus using 
VIO and additional soft modules embedded with the soft core. 
Two ARM 7 Micro controller boards with Integrated CAN controller and trans- 
ceivers (Node KE and OL in figure |2j. These boards are used as CAN Receiver 
Nodes for further verification of the messages sent by the CAN soft core. The other 
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utility of these CAN nodes are to induce extra or erroneous bits which is a require- 
ment of few test cases as mentioned in 0. 

The figure |2] demonstrate a 2 node Soft Core, it can easily be extended to n number of 
nodes considering the protocol limitations. 

3.2 Software 

1. Xilinx ISE ll2"Tll for soft-core programming, synthesis, routing and programming 
the FPGA. The ISE is a complete IDE for FPGA development and contains some 
extra features like power analysis, optimal routing and timing analysis to name a 
few. 

2. Chipscope Pro is used as analysis tool. The VIO core is used to generate and control 
different bit patterns. These bit patterns can be synchronous or asynchronous. 

3. The Keil uVision 3 IDE ||22| with free ARM tools C compiler was chosen for 
programming and debugging the Microcontroller boards. 

As can be seen, the test bed has been made using COTS hardware and also taking in 
care the structure of the Test Plan given in the ISO 9646- 1 . The Test Bed consists of two 
instances of the IUT, the main purpose of using the second IUT is to generate errors on 
the CAN bus and special conditions which were either the pre-requisite for a test case or 
generating special bit stream during a test for verifying the behavior of the IUT, hence 
the second instance of IUT is moreover working as the LT in reference to the IS09646- 1 
TP 

The ARM boards with integrated CAN Controllers were used either as receivers/ 
transmitters to verify the conformance of the IUT with widely used CAN controllers, 
and were also employed to generate bit errors on the CAN bus using an interrupt gen- 
eration mechanism. This scheme is highly synchronized as the bit inversions were done 
at the specific point where it was required; the methodology employed for test pattern 
generation is described in the next Section. 

3.3 Test Pattern Generation 

When using pattern generators test vectors are required to be first stored, and are sent on 
the CAN bus only when required thus putting the IUT in different states and allowing 
its behaviour and responses to be analysed. In our proposed test bed we have used FPGA 
based pattern generation, which is not only economical as no extra price was added to 
the test setup but also it is added as a separate Verilog [23] module to the main CAN 
Core and it will work non-intrusively as this cod is part of the CAN core which is taking 
the role of an LT. This helped us to accurately produce special conditions; for example 
in a test case it was needed to produce extra dominant bits on the CAN bus after an IUT 
working as a transmitter send an Error Frame 1 2 ] . This test pattern was easily achieved 
by modifying the Verilog module for Error Flag generation to produce extra dominant 
bits, as illustrated by the code fragment: 
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Fig. 3. Transmitter snapshot for Overload frame test case 



reg [3:0] Error_Flag_Counter ,- //changed from reg [2:0] 

always @ (posedge Clock or po sedge Reset) 
begin 
if (rst) 

Error_Flag_Counter <= 4 ' dO ; 

else if (Error_Frame_End | Error_f rame_Start) 

Error_Flag_Counter < = #delay 4 ' dO ; / /changed from 3 ' d7 
else i f ( Error_Frame & Transmi t_Instance &Error_Flag_Counter < 4 ' dll ) 

Error_Flag_Counter < = #delay Error_Flag_Counter + 1 ' bl ; 
end 

always @ (Error_f rame or Error_Flag_Counter ) 
begin 

if (Error_f rame) 
begin 

if (Error_Flag_Counter < 4 ' dll) / /changed from 3 ' d7 
begin 

if (Node_Error_Passive) 
Tx_CAN = 1 ' bl ; 
else 

Tx_CAN = 1 ' bO ; 
end 
end 
end 



4 Test Cases 



The proposed test facility was employed to test the CAN conformance of the custom cre- 
ated CAN soft core, written in Verilog. As the number of total number of test cases to 
consider in any single CAN conformance test plan is numerous, it is beyond the scope of 
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the current paper to present comprehensive test results; such test results are available in 
the form of technical report [ 1 8 1 . However, in this Section we will present two test cases 
that help highlight the main features of the proposed facility. Both tests were carried out 
successfully, and are described in the following two Sections. 

4.1 Overload Frame Management 

This test is apart of the Overload Frame Management class 1 2 1 . This test verifies that an IUT 
will be able to transmit a data frame starting with the identifier and without transmitting 
SOF, when detecting a dominant bit on the third bit of the intermission field. This test 
involves two instances of the IUT and the ARM7 Micro controller boards. The test will be 
setup using the following organization: 

1 . Both of the IUT's must be in default state ready fortransmission orreception according 
to the setup sent by the Host Controller. 

2. The IUT acting as the Transmitter is set to transmit two data frames as programmed 
in the Host processor. 

3. The Receiver IUT will be set to request an Overload frame after reception of the first 
frame. 

4. After the completion of the Overload Frame on the third bit of the Intermission field 
(Normally the Intermission field is a sequence of three Recessive bits) is set to domi- 
nant by the Fault injector node i.e. K.E. 

5 . The transmitter must not consider it as a bit error and shouldn' t send a Dominant level 
SOF and consider the dominant bit of the Intermission field as the SOF. 

6. Normal reception of the message should take place. 

This test was successful with desired results as stated in the purpose of the test; the obser- 
vation on the transmitter node from the Chipscope - shown in figure[3] is as follows: 

1. Left of Marker T' The Tx_state flag is high indicating ongoing transmission, 
Receive_state JData and ACK_DELIM indicating a successful transmission while the 
node is error active. 

2. At Marker T' there is an error on the Receive_state_intermission field generating an 
overload frame with Overload Flag of six dominant and Overload delimiter of 8 bits 
as can be seen by the count of sample point. 

3. After the Overload frame an intermission field signal can be seen at the Marker 'X' . 

4. The third bit of intermission field is is a dominant bit as can be counted between Mark- 
ers 'X' and 'O' the number of sample points is 2 and the third sample point is a dominant 
bit. 

5. Just after the Marker 'O' we can see the Receive_State_ID [10:0] going high without 
any SOF. The Identifier first 4 bits are dominant as required by the Test case. 

4.2 Suspended Transmission Field Test 

The purpose of this test is to verify that a Passive state IUT acting as a transmitter doesn't 
transmit any frame before the end of a suspended Transmission field following an Error 
Frame. 
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The setup for this test is to put the Transmitter IUT into Error Passive state and then 
introduce a bit error during transmission. The IUT will send a Passive Error Flag which 
is overwritten by Active Error Flag from participating CAN nodes, the IUT will receive 7 
dominant bits as the Error delimiter. After the completion of the Error Flag, the IUT will 
send the pending data frame once the Suspended Transmission Field is over. 

This test was successful with desired results as stated in the purpose of the test; the 
observation on the transmitter node from the Chipscope - shown in figure[4] is as follows: 

1. Left of Marker "F The Tx_state Flag is high indicating on going transmission when 
a bit error set the node to send an Error _Frame, the node is Error Passive. 

2. Between Marker 'X' and 'O' there is no transmission when the Tx_Suspend_state is 
high which is indicated by the TX_State low status although in case of error recovery 
the re-transmission of the message is carried soon after the intermission field. 

3. As the Tx_Suspend_state is set to low after 8 bit times then the SOF is indicated by 
CAN .Receive and CAN .Transmit signals which changes to dominant state. 

4. Rest of the Receive state signals indicates the re- transmission. 

5 Comparative Study 



This Section presents a cost and flexibility comparison between conventional CAN con- 
formance testing hardware and software with the approach that has been discussed in this 
paper. The first observation is that the next facility does not require expensive CAN PC 
interface cards [24 25 ] which are normally required for CAN conformance testing Il26ll . 
These cards are used to capture CAN bus data to analyse the internal status of different reg- 
isters and to log the events; these cards not only required the hardware but also specialized 
software ll27ll along with interface cables which can also add to the cost and complexity of 
the setup. In our proposed implementation we can analyse the internal state of CAN IUT 
directly using Chipscope, and also by using the Keil uVision 3 IDE. This IDE can run an 
online debug on the COTS ARM boards, which are significantly cheaper in comparison 
to specialised CAN interface Cards. The Keil IDE can display the status of all the CAN 
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registers hence providing us useful information in verification of the status of the CAN bus 
and the data being transmitted and received. In addition, there are several key advantages 
of our proposed test bed using Chipscope over hardware logic analyser systems: 

1 . The standard bench analysers doesn't show enough signals as required in case of CAN 
conformance as illustrated in section 4. There are Logic analyser systems which can 
show large number of signals simultaneously with large data widths [28 1 but there 
prices are 10 times more than Integrated Logic analyser. 

2. Normal Bench analysers can show Mega samples 1 29 28 ] , while the Chipscope is lim- 
ited to a Sample width of 1 6K, we overcome this problem by using Digital clock Man- 
ager [ 30 1 which can divide or multiply the system clockby 'n' times, the board we used 
in our system can divide the system clock by 16 times hence we were able to capture 
16 times more sample than on system clock which can easily capture 3 to 4 complete 
CAN messages in a single trigger. 

3. Additional probes with wide numbers of I/O pins are required to interface with the 
Logic analysers while Chipscope can carry magnitude of these signals using a simple 
JTAG cable, although there are few solution like Agilent's FPGA trace port 1 3 1 ] which 
use a simple interface to analyse multiple signals but it also requires a specialized 
hardware and Chipscope pro tool. 

4. Not only all I/O signals are accessible through Chipscope but also internal wires can be 
traced 1 32 ] which are really helpful in Conformance testing specially when setting up 
triggering conditions we have lot more options to setup a trigger condition for example 
in the test cases discussed it is really easy to setup a trigger condition to wait for an 
Error Frame flag signal goes to high to analyse an error condition, while for external 
Logic analysers only I/O signals are available. 

6 Conclusions and Future Work 

In this paper we have presented an approach to utilize Virtual I/Os and Integrated Logic 
Analysers to perform CAN conformance testing in accordance with the ISO standards. 
It has been shown that the facility is capable of performing the full range of test required 
and specially related to CAN bit timing tests conforming to the relevant CAN standard. 
In conclusion, this facility can be assembled and used for a fraction of the cost of a regu- 
lar test facility for CAN conformance. A full list of the how each individual test may be 
implemented when using a facility such as this has been described in 1 1 8 1 . 

As a final note, it can be seen that test facility that has been described is not restricted 
to the CAN protocol, and with suitable modifications can be used to test conformance of 
many alternate network protocols, for example 1 33 1 . 
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Abstract. Bilateral teleoperation system are prone to instability coming out from 
the time-delay introduced by the indeterministic communication channel. This 
problem has been subject of intensive research under the assumption of non-equal 
master-slave teleoperators, however, dynamical similarity of master-slave robots 
can always be induced by proper tuning of feedback gains in the controllers, 
based on the physical ratio of both robots When we consider a linear Dynam- 
ically Similar Teleoperation System (DSTS), there arises advantages, observed 
heuristically in several experiments [1] or analytically in |10|. In this paper, a 
simple yet important parametrization trade-off of feedback control gains and per- 
formance teleoperation parameters is presented that allows an easy way to achieve 
stability or even also passivity. The analysis is based on an an impedance control 
scheme reported in 1 2 1 when the time-delay in the communication channel is con- 
sidered constant but unknown. Stability and passivity limits are obtained based 
on Llewellyn's and Raisbeck's criteria. 

This result explicitly suggests clearly guidelines among key factors, such as 
time delay, desired velocities and feedback gains in terms of the scaling pa- 
rameters, arises a clear advantage when dealing with dynamically similar sys- 
tems. This explains why the transparency of the teleoperation system, also stud- 
ied under the same control law in 1101 is improved by augmenting/reducing the 
dynamic/kinematic scaling factor, for given desired frequency, time delay and 
feedback gains. 

Keywords: Teleoperation, Haptics, Parametric design. 



1 Introduction 

A bilateral teleoperation system is composed of a master robot and a slave robot, with 
a human operator commanding the master robot in order to produce the desired posi- 
tion and/or contact force trajectories for the remotely located slave robot, depending 
whether the slave tool is in contact or not. In turns, the slave robot follows these trajec- 
tories and touches the environment with contact forces that have to be sent back to the 
master robot to induce the touching sensation to the operator. In this way, a force/force- 
position control law is implemented in the master/slave stations with a communication 
channel introducing delayed position and forces signals |4). It is well known that the 

J.A. Cetto et al. (Eds.): Informatics in Control Automation and Robotics, LNEE 85, pp. 143 |l55.| 
springerlink.com © Springer- Verlag Berlin Heidelberg 201 1 
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Fig. 1. Basic Bilateral Teleoperation Scheme 

source of instability of bilateral teleoperation system appears because the time-delay 
introduced by the indeterministic communication channel is not passive [8|. Then, the 
limits to achieve human operator stable interaction with the slave robot, placed at a 
remote location, through the master robot, becomes an issue, in particular successful 
teleoperation requires a certain degree of transparency (Manipulation of the slave robot 
without any dynamics in between) and/or telepresence (Sensation of being physically 
in the remote environment). Limits of stable and passive interaction for DSTS subject 
to time-delay in the communication channel is studied in [10], and also advantages of 
dynamically similar 

In this paper, motivated by the empirical observation that a DSTS are easier to handle 
and the previos results presented in HOll . it is argued that dynamic and kinematic simi- 
larity introduce a clear trade-off of some of the aspects of telepresence and transparency, 
that not only depend on the simultaneous convergence of the position and force error 
between the master and slave robot but also on the subjective cognition of the human 
operator of being there, which in turns depends, among other aspects, on visual aids of 
the remote environment, kinesthetic coupling, the ability to deal with delayed signals 
and the man-machine interface. 

Then it is reasonable to expect a simple trade-off of these aspects. Preliminary ana- 
lytical results are presented in 1 10], based on an impedance control scheme [2], when 
the delay is considered unknown but constant. 



1.1 Contribution and Organization 

In this work the trade-off discussed in the previous section is presented in a parametrical 
way that allows better understanding of the stability /passive limits in the chosen set for 
the scaling factors and the imposed closed-loop impedance parameters. 

Our basic hypothesis is that as long as the human perceives linear correlated vari- 
ations in both teleoperated robots, he can improve the command of the closed-loop 
bilateral teleoperation system since spatial and temporal attributes of the visual remote 
location and kinesthetic coupling will vary linearly without distortion. So cognitively, 
the human can quickly learn to command the task with greater dexterity. Additionally in 
this paper we deal with unknown time delay so we design a novel controller to deal with 
unknown constant time delay El, HI. A computed- torque controller is employed in the 
master station and a computed torque second order sliding mode controller in the slave 
station is proposed to produce a desired impedance in closed loop. Then, stability the- 
ory is used to analyze the closed-loop stability and passivity properties and therefore the 
limits of human-teleoperation ifTOl scaling factors. The contribution of this work lays 
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on the fact that trade-off between these scaling gains can be achieved easily by looking 
at the stability/passivity limits when expressed in an appropriate parametric way. 

Hence a review of DSTS with constant unknown delay is presented in Section [2] 
while controllers are explained in Section[3] Stability and passive analysis using Llewe- 
lyn (7J, and Raisbeck [9] criteria, and their parametrization are presented in Section [4] 
The full relationship and trade-off of the system parameters that can be used for design 
purpose are given in Section|5] Final conclusions are given in Section[6] 

2 Dynamically Similar Delayed Teleoperation System 

A dynamically similar teleoperation system [3| has constant scaling factors which re- 
late kinematic and dynamic parameters of the master and slave robots. This similarity 
between the systems is poorly understood so far, since there is not theoretical apparent 
evident advantage to work out with bilateral system, despite some analysis reported in 
[ 6 ] , where the advantages has not been addressed properly in terms of explicit trade-offs 
of feedback gains, system parameters, desired trajectories and time delay. We surmise in 
this paper that DSTS improves significantly the ability of humans operating the master 
teleoperator to carry out efficiently teleoperation tasks at remote environments, when 
there is an unknown time delay involved in the communication channel. 

Impedance control has been explored in [ 1 1 to enforce a desired impedance dynamics 
in closed loop in order to program arbitrarily the desired impedance parameters. This 
closed-loop linear dynamic allows to model the entire system as a 2-port network to re- 
late the force and flows of input and output, respectively, by an impedance matrix or an 
hybrid matrix. This matrix can be used to describe the stability of the entire system us- 
ing the Absolute Stability Theory |5 1, where a tight relationship between output scaling 
factors and impedance parameters can be found to give sufficient conditions on stability. 
However, when dynamic scaling factors are introduced, the master and slave robot dy- 
namics are related by constants, either in kinematic and/or dynamic parameters, thus, 
a sort of advantages can emerge from this relation since a single feedback parameter 
appears. In this paper, we offer an analysis in terms of both the Llewelyn's criteria and 
Raisbeck's criteria, and verify its real time performance, which demonstrates a clear 
and intuitive trade off in terms of scaling factors of the DSTS. 

2.1 Dynamically Similar System 

Consider the dynamics of a linear teleoperation system consisting of two n-DoF manip- 
ulators decoupled systems as follows 

M s x s + B s x s + K s x s = F sc + F s (2) 

where Xi and Xi denote acceleration and velocity of the robot ^respectively; F mc , F sc 
are the control force inputs and F m , F s are external forces induced to the master and to 
the slave systems, respectively. The dynamic parameters Mi, Bi and Ki are the inertia, 
damping and stiffness, positive coefficients, of the systems; with i = m for the master 
and i = s for the slave. 



146 E. Olguin-Diaz et al. 

Let Kk > be the kinematic scaling factor of the slave configuration with respect to 
the master's, such that 

Then also holds x s = K^x s and x s = KkX m . 

Definition 1. The system expressed by (CJ-dUl is said dynamically similar after the co- 
ordination Q if there exists a scalar £ > such that 

Ti r •• i—t • r r S X S I J-* s "- 1 S I Ji s X g 

IVl ra X T! i -\- D rn X rn ~T ^m^rn j. v*) 

Thus, the dynamic parameters {Mi, Bi, Ki) of the master and those of the slave in 
(fl}-© are linearly related by 



X s = ^X m = (X mi X % = { B. damping o) 



mass 
damp 
stiffness 



where the constant 

S K 
takes account of both kinematic and dynamic scalings. 

The apparent advantage of this dynamic relationship has not been well explored in 
the context of teleoperation with unknown constant time delay, though there are a lot of 
heuristical intuition which leads us to conclude that these systems would allow greater 
kinesthetic coupling with greater manipulability dexterity as consequence. To this end, 
it is introduced an impedance control system [ 2 ] to enforce a 2-port closed- loop desired 
linear system. 



3 Impedance Control Law 

3.1 1 DoF Teleoperation System 

Similarly to (fl}-©, let a 1 DoF master/slave teleoperation system be modeled as a 
mass-spring-damper system, where external master force F m on the master is nothing 
but the human commanding force Fh and the external slave force force F s stands as the 
environmental contact force F e , then (Q]!-© becomes 

M s x s + B s x s + K s x s = F sc - F e (7) 

The negative sign in F e appears due to fact that this force is considered to be the force 
applied by the robot to the environment (in opposite direction as the force induced by 
the environment to the robot's end effector due to Newton's 3 rd Law), which is the 
same that is read by the force sensor in the robot and is available for force feedback. 
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3.2 Impedance Control Law for the Master 

For completeness, the control law [2| is introduced here. Consider the following master 
controller 

The control law (£Sj when applied to the open-loop system © gives rise to the following 
close-loop impedance equation for the master robot 

M m x m + B m x m + K m Xm = F h - K f F* v (9) 

where the positive constants M m , B m , K m are the desired values of inertia, damping 
and stiffness for the master robot. The constant gain Kf is the force scaling factor that 
makes the user (Fh) to feel a portion of the desired environmental contact force Ff v . 
The last is defined as the most recent real one in the slave, that can be read in the master, 
after the telecommunication transmission as 

F* y =F e (t-T s ) (10) 

with T s being the unknown but constant time-delay from the slave to the master station. 
That is, the master impedance control law enforces a desired impedance (0 in closed- 
loop, whose parameters are chosen by the user depending of a specific task, such that: 

- If the slave robot is not touching the environment, F^ v = 0. Then Q becomes a 
mass-spring-damper system driven solely by the human force F^. Notice that in 
this case the controller is in position impedance mode. 

- If the slave robot does touch the environment, F^ v > 0. Then (O becomes a mass- 
spring-damper system driven by force error Fh — KfF^ v . In this case, actuators 
in the master station makes the human to perceive a scaled contact force equal to 
KfF^ y . Notice that when the slave is contact, the master control is in force control 
mode. 

In this mode the human could virtually recreate, cognitively, the surface of the 
object that the slave is touching according to this vector, through kinesthetic sensa- 
tions and the visual information coming also from the slave station. 

3.3 Impedance Control Law for the Slave 

In order to achieve the desired effects in the master robot, the slave robot must be 
controlled in both impedance position and force control modes, according to the contact 
regime. Similarly to the master controller, the objective in the slave station is to impose 
a desired impedance to the slave robot 

M s x s + B s x s + K s x s = -F e (11) 

where positive M s , B s , K s are the desired inertia, damping and stiffness for the slave 
robot, respectively. The position tracking error a; s is expressed as follows 

x s x s iS-pX^^ yLz.) 
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where the constant gain K p is the position scaling factor that makes the slave to move a 
portion of the desired task space position, which in turn is defined as the most recent real 
one in the master, that can be read in the slave, after the telecomunication transmission 
as: 

being T m the unknown, but constant time-delay from the master to the slave station. 
Now, consider the following control law for the slave robot, presented in [2]: 



M 
--J- (B s x s + K s x s + F e + K % a) + F e + B s x s - K„f2 

M s y ' 




+MSK.M,- 1 (if - K t Ff - B m x% - K m x d J) 


(13) 


M s x s + B s x s + K s x s + F e 


(14) 


/ sgn{/ e (r)}dT 


(15) 



Q 



-fij-(j Ie(T)dr + J J sgn {I e (r)} dadr) (16) 



where the desired force reference F h y is defined as the delayed master's operator input 
force: 

F* v = F h (t-T m ) 

and Ffv as in JTDTt. 

The proposed sliding surface I e in dT4l is naturally proposed out of the desired close- 
loop ( fTTb . because it is desired that this function becomes an attractive convergent man- 
ifold. That is if (TBI) converge to zero, then ifTTI) appears and the human would perceive 
the desired impedance at the robot and would be able to control the slave robot at will. 
Finally the high order sliding surface Q is built as a function of the sliding surface I e 
such that substitution of dT3t-(fT6t into (Ql gives rise to the closed-loop error equation 
for the slave robot: 

h = -pn (17) 

where j3 = jf- > and dT7] ) is Lipschitz. Consequently, all closed-loop signals in the 
slave station are bounded, enforcing exponential convergence of Q — > 0. Therefore, 
this chain of implications means that a second order sliding mode is enforced, and a 
sliding mode arises, at I e = 0, which means that (fTTI l arises in finite-time. 

Remarks: 

1. Notice that the feedforward term Ff v in (TT~3l > allows control without delayed mea- 
surement of F e . In any case, notice that F e , being measured at this slave station is 
available at any time. 

2. Notice that the gain K g is a new control variable that weights the extended error 
variable fi. 

3. A closer analysis shows that the slave impedance control law enforces a desired 
impedance in closed-loop whose parameters are chosen by the user depending of a 
specific task, such that: 
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- when the slave robot is not touching the environment, F h v — 0,F e = and 
(fTTb becomes an unforced mass-spring-damper system such that x s — > and 
the slave tracks the desired delayed position and velocities of the master. Notice 
that when the slave is not in contact, the slave control is in position impedance 
control mode. 

- when the slave robot is touching the environment, F e > and (fTTb becomes 
a mass-spring-damper system driven by the slave contact force F e . In this 
case, actuators in the slave station make that the slave robot maintains contact 
(| x s | > 0) while F e stays around F h v . Notice that when the slave is in contact, 
the slave control is in impedance force-position control mode. 



4 Delayed Teleoperation Stability 

Last section presented the control law for each station, master and slave, and their in- 
dividual stability analysis. However, the overall teleoperation stability system must be 
also addressed. 

With the desired impedance imposed by the controllers ([8]) and (fT3b-(fT5b. the closed- 
loop dynamics ^ and ( fTot can be modeled as a 2-port network. Transforming this 
dynamic into the frequency domain and doing some algebra, closed-loop system can be 
represented as 



F h 
V a 



hn(s) h 12 (s) 

tl2l(s) h 2 2{s) 



V m 

-F K 



H(s) 



-F K 



(18) 



where V m and V s are the velocity signals, in Laplace domain, of the master and slave 
respectively and H(s) is the so called Hybrid Matrix. Using the dynamic similarity 
scales (01 the hybrid matrix is built out from elements depending on the master's desired 
impedance parameters as follows 



H(s) = 



M m s 2 + B m s+K„ 



-Kje 



(19) 



C M m s 2 +B m s+K n 

which is fundamental to carry out the implications of a unique dynamic scaling factor. 



4.1 Absolute Stability 

The absolute stability definition for a 2-port systems [2| is stated as: 

Definition 2. Llewellyn's Absolute Stability Criteria for 2-port Systems: A two-port 
system ( I78D -(I79D is absolute stable if it does not exist a set of impedances for which the 
entire system become unstable. If the network is not absolutely stable, it is potentially 
unstable. By the conditions of the Llewellyn's criteria a 2-port network is absolutely 
stable if and only if 

1. hn and h<xi have no poles in the right half plane 

2. Any poles ofh\\ and h 2 i on the imaginary axis are simple with real and positives 
residues 
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3. For all real values of to 

(a) Re{/m(ju;)} > 

(b) Re{/i 22 (jo;)} > 

(c) 2Re{/m(juO}]Re{Miw)} - ^{h 12 h 21 {juo)} - \h l2 h 21 (jcj)\ > 

Notice that since the human operator is physically holding the master robot with his 
hand, it is imperative to ensure stable behavior. Thus it is required to guarantee the 
fulfillment of previous Definition. To this end, when using ( fT9t , notice that conditions 1 . 
and 2. are trivially satisfied with positive impedance parameters. For the third condition, 
with s — ju>, a) and b) are also trivially satisfied while c) becomes: 

Definition 3. The stability condition for delayed dynamically similar teleoperation 
systems: 

K p K f (l - cos{(T m + T s )u}) < 2v{w)/C, (20) 

where the function v{uj) is defined as: 

n w ) = -rz = — 72 — = ( 21 ) 

(K m -M m Lj2) +(B m ui) 2 

withO < v(uj) < 1. 

A conservative result, independent on the communication's time-delay channel, can 
be expressed using the bounds on the left-hand-side of condition ( f20b : 

< ( cos{(T m + T s )uj} + l)K p K f < 2K p K f 

Then a delay-independent stability condition can be stated with the next expression: 

CK p Kf < v{uo) (22) 

4.2 Passivity Analysis 

Passivity is a powerful criteria to analysis the energetic coupling of a closed loop sys- 
tem, a more conservative implication in comparison to Lyapunov stability criteria, how- 
ever since the human operator is physically coupled with a typically mechanical system 
in closed-loop, it is important to analyze the passivity of the closed-loop system. 

A two-port network is said to be passive if for all inputs of energy, the output energy 
is equal or less than the input energy. If the network is not passive, it is active. Rais- 
beck's passivity criterion is used to determine the passivity of the system. 

Definition 4. Raisbeck Passivity Criteria for 2-port Systems: It is said that a 2 port- 
network is passive if and only if 

1. The elements hij of the hybrid matrix ( 1791 ) have no poles in the right half plane 

2. Any poles of the elements of the hybrid matrix on the imaginary axis are simple and 
their residues rij satisfy the following conditions, for all real values ofu>, 

(a) r n > 0, 
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(b) r 22 > 0, 

(c) r Y1 r 22 - r 12 r 2 i > 

3. The elements hij of the hybrid matrix must satisfy the following condition: 

4Re{/ lll }Re{/ l22 } - [Re{/li 2 } + Re{h 21 }] 2 - [lm{h 12 } - lm{h 21 }} 2 > Q 

In a similar way, the first two conditions are satisfied with positive impedance parame- 
ters, while the third is expressed as: 

Definition 5. The passivity condition for delayed dynamically similar teleoperation 
systems: 

K 2 p + K) + 2K p K f cos{(T m - T s )co} < 4i/(w)/C (23) 

A conservative result, independent on the communication's time-delay channel, can be 
expressed using the bounds on the left-hand-side of condition < f23b : 

(K p - K f f < K 2 p + K) + 2K p K f cos{(T m - T s )u} < (K p + K s f 

Then a delay-independent passivity condition can be stated with the next expression: 



K p + K f <2J^-± (24) 

5 Design Parametrization 

5.1 The Bounding Function v(u)) 

The function v, defined in (l2TT i. that is used in the stability and passivity conditions, 
express the bounds of those conditions in terms of a frequency u>. This frequency repre- 
sents the spectrum of the transmitted signals for- and backwards in the communication 
channel. So at the end, this bounding function, and as consequence the stability and 
passivity conditions, depends also in the shape and richness of the transmitted signals 
for teleoperation. 

It is always possible to filter the transmitted signal and cut-off noisy remanent fre- 
quencies beyond some allowed bandwidth. To select this parameter the bounding func- 
tion v can be rewritten in terms of only two impedance parameters instead of three. This 
is because the mass of the desired dynamics M m is chosen to be very close to the actual 
mass of the system while the chosen parameters are the damping B m and the rigidity 
K m . Then if close-loop natural frequency io n and the damping ratio £ are chosen as the 
new parameters, defined as follows: 

V Mm 2^/K m M m 

and the frequency is normalized by the close-loop natural frequency, the bounded func- 
tion is rewritten as: 

u{u>) = -j (26) 



(w/w„) 2 ) +4?(u>/u, n y 
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Fig. 2. Bounding function v(ui) for different values of damping ratio £, as function of the nor- 
malized frequency 
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Figure |2] shows the shape of the bounding function v for different values of damping 
ratio £. From d2ol it can be calculated that the function gets its maximum at the natu- 
ral frequency u) = U) n regardless of the damping ratio. The extreme solutions for the 
damping ratio, i.e. £ = (which yields v = 0, \/uo ^ uj n ; with a singular value v = 1 
at w = w n ) and £ = oo (that yields to v = 1, Vw > 0) are purely hypothetic cases. 
The first because there is no possible physical way to achieve a non dissipative me- 
chanical system (there is always some kind of friction) for the closed-loop slave/master 
robotics systems. The second because it implies the use of a very large value of close- 
loop damping factor B m that would render the closed-loop behavior extremely slow 
and energy consuming (and still not guarantying stability nor passivity). A more prac- 
tical choice is that of picking a damping ratio in the vicinity of the critical value £ = 1 
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(critically damped case). For these values, it can be seen in Figure [2] that there is an 
"acceptable" bandwidth: 



L0 b 



A 



1 



-w n ,aw„ 



centered at the chosen close-loop natural frequency u) n . The size of this bandwidth, i.e. 
the value of a, depends on two factors. One is the chosen damping ratio £, but the other 
is an acceptable upper-bound v of the bounding function u(u>). Table [TJ shows values of 
the bandwidth parameter a(i>, £) for given values of acceptable bounds v and damping 
ratio £. The bolded values in the table are also shown in Figure [2] On the other hand, 
Table|2]shows values of acceptable bounds of D(a, £) for given values of the bandwidth 
parameter a and the damping ratio £. The bolded values are also shown in Figure[2] 
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Fig. 3. Force/position scaling gains plot with stability and passive limits 
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5.2 Scaling Gains 

Stability and passivity conditions expressed by d22l and (l24l respectively can both be 
expressed in a phase plot of the scaling factors for position and force in the teleoperation 
scheme. This plot is shown in Figure[3] 

The border curves in Fig. [3] are parametric dependant on the close-loop impedance 
desired parameters (mass, damping and stiffness) and hence by the induced natural 
frequency and damping ratio. Also they have dependance on the acceptable bandwidth 
which is parameterized also by upper bound v or frequency factor a, but more important 
they depend on the dynamic similarity scale factor. These borders defines the areas 
in which the values of the position and force scaling gains shall be placed in order 
to preserve stability or even more restrictively passivity. The borders are somewhat 
defined by the physical construction of the system by the dynamic similarity scaling 
factor but they can be replaced by changing the impedance parameters or the acceptable 
bandwidth transmission. 

6 Conclusions 

Using a novel impedance controller and advanced stability tools, precise conditions to 
guarantee stability, even in harsh conditions, is proposed for dynamically similar bilat- 
eral teleoperation robotic system. In this case, this system depends on a constant pa- 
rameter, which relates explicitly and clearly a trade off between stability /passivity with 
scaling gains (position, force and dynamic similarity) and acceptable channel band- 
width transmission. 

The controller enforces convergence in finite time due to the sliding surface, which 
is nothing but the impedance equation, thus the closed-loop system dynamics is en- 
tirely governed by the desired controlled equation. This yields useful boundaries to 
vary impedance, scaling parameters and frequency, in terms of the bounded time delay, 
which in turn allows to introduce a desired performance criteria in terms of surrounding 
physical conditions. This result seemingly allows to establish a simpler methodology to 
design dynamically similar teleoperators with desired forward and backward scaling 
gains, on a given desired performance in realistic conditions. 

Therefore, the consequences of introducing a dynamic similar system in teleopera- 
tion, are: 

1. scaling factors of position, force, and dynamical similarity ( are critical for the 
absolute stability or the passivity performance of the system. 

2. The introduction of a similarity factor £ offers a simpler way to analyze the design 
and to tune the system performance. 

3. The similarity factor £ improves the design methodology of teleoperators based 
in impedance controllers. That is, there is a clear trade-off of all important pa- 
rameters of the system: depending on impedance and the desired performance 
parameters(w n , £, M m , B m and K m ) and time delays, the scaling factors (posi- 
tion K p , force Kf and dynamic similarity Q can be found. 

4. From (120b . the scaling factor £ allows bigger margin on other parameters, thus 
the opportunity to improve the performance based on the physical structure of the 
teleoperation system. 
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5. Due to the fact that both master and slave impedance parameters are related by 
this factor, the whole set of parameters can be expressed in terms of each other, 
reducing the number of parameters implied in the design process, making it easier 
to establish a performance limit. 

Then absolute stability is a more relaxed stability criteria than passivity. The passivity 
condition is necessary to assure a complete energetic stability performance of the closed 
loop system. Thus, we can choose the dynamic scaling factor in order to have a greater 
upper and lower bounds to vary the other factors of position and force, affecting also 
passivity. However the last can be chosen to set the performance parameters because it 
is of primary interest to maintain passivity since the human is physically holding the 
mechanical master robot. 
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Abstract. Path-planning is a key technique in visual servoing, as it allows one 
to take into account various constraints and optimize a desired performance. This 
paper addresses the problem of planning image trajectories to be followed by us- 
ing an image-based visual servoing controller. The proposed technique consists of 
parametrizing the camera trajectory in the 3D space via polynomials, and by im- 
posing constraints satisfaction and performance optimization via the square ma- 
trix representation (SMR) and linear matrix inequalities (LMIs). Examples in the 
absence and in the presence of uncertainties illustrate and validate the proposed 
methodology. 



1 Introduction 

Visual servoing is an important area of robotics that concerns the closed-loop control 
of robotic system with visual feedback. The problem typically consists of steering a 
robot end-effector from an unknown initial location to an unknown desired location by 
using the visual information provided by a camera mounted on the robot end-effector. 
The camera is firstly located at a certain location, called desired location, and the image 
projections of some object points are recorded. Then, the camera is moved to another 
location of the robot workspace, from which the same object points are visible, and 
whose relative motion with respect to the desired location is unknown. The problem 
consists of reaching the desired location from this new location, which is called initial 
location. See for instance [ 1 ] and references therein. 

The procedure just described, known as teaching-by-showing approach, has numer- 
ous applications, for instance in the industrial manufacture for the construction of com- 
plex components such as parts of a ship, where its function consists of allowing a 
robotic arm to grasp and position tools and objects. Other applications are in surveil- 
lance, where a mobile camera observes some areas of interest such as the entrance of 
a building in order to identify people, and in airplane alignment, where the system to 
be positioned is represented by the airplane that has to be aligned with respect to the 
runway in order to land. Also, the teaching-by-showing approach finds application in 
surgery, where an instrument is automatically guided to the organ to operate, in naviga- 
tion, where a mobile robot has to explore a scene, and in dangerous environments such 
as nuclear stations and spatial missions, where humans should be replaced. 

J. A. Cetto et al. (Eds.): Informatics in Control Automation and Robotics, LNEE 85, pp. 159 j-172.| 
springerlink.com © Springer- Verlag Berlin Heidelberg 201 1 
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In last years, various methods have been developed for addressing the teaching-by- 
showing approach. Some of these methods have proposed the use of the camera pose as 
feedback information (known as position-based visual servoing, see e.g. |2|), definition 
of the feedback error in the image domain (known as image-based visual servoing, see 
e.g. O), use of both camera pose error and image error (known as 2 1/2 D visual 
servoing, see e.g. H), partition of the degrees of freedoms 13, switching control for 
constraints satisfaction [6 7], generation of circular- like trajectories for ensuring global 
stability and minimizing the trajectory length [8|, use of complex image features via 
image moments [9|, global motion plan via navigation functions IfTOll . enlargement of 
stability regions [11], and model-less control lfl2l . 

Path-planning strategies have also been proposed in order to take into ac- 
count multiple constraints and optimize a desired performance. See for instance 
II13I14I15I16I17I18T 19I and references therein. These methods generally adopt potential 
fields along a reference trajectory in order to fulfill the required constraints, in particular 
the potential fields do not affect the chosen reference trajectory wherever the constraints 
are not violated, while they make the camera deviating from this path wherever a con- 
straint does not hold. 

This paper addresses the problem of planning image trajectories to be followed by 
using an image-based visual servoing controller. The trajectories connecting the initial 
location to the desired one are parametrized through polynomials by estimating the ini- 
tial camera pose and the position of the object points. Typical trajectory constraints and 
guaranteed level of performance are formulated in terms of positivity of certain poly- 
nomials, which is imposed by using the square matrix representation (SMR) and linear 
matrix inequalities (LMIs). Then, the image trajectory of the object points along the 
planned camera trajectory is tracked by using an image-based visual servoing controller. 
Examples in the absence and in the presence of uncertainties illustrate and validate the 
proposed methodology. 

This paper extends our previous works (T7\ by exploiting LMIs for imposing con- 
straints, and [18] by introducing alternative techniques for converting polynomial 
inequalities into LMIs. 

The organization of the paper is as follows. Section [2] introduces the problem for- 
mulation and some preliminaries. Section [3] describes the proposed approach for plan- 
ning image trajectories. Section [4] illustrates the simulation results. Lastly, Section [5] 
concludes the paper with some final remarks. 



2 Preliminaries 

In this section we introduce some preliminaries, namely the notation, problem formu- 
lation, and a tool for representing polynomials. 

2.1 Notation and Problem Formulation 

Let us start by introducing the notation adopted throughout the paper: 
- R: real numbers space; 



des j.des~[ \ ) 
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- 0„:nxl null vector; 

- I n : n x n identity matrix; 

- 1 1 v 1 1 : euclidean norm of vector v . 

We consider a generic stereo vision system, where two cameras are observing a com- 
mon set of object points in the scene. The symbols F m% and F des represent the frames 
of the camera in the initial and desired location respectively. These frames are expressed 
as 

rpini / Tyini a-ini \ 

rpdes / Tides ±des\ 

where W m ,R des G R 3x3 are rotation matrices, and t ml ,t des G M 3 are translation 
vectors. These quantities R m% , R des , t m% and t des are expressed with respect to an 
absolute frame, which is indicated by F abs . 

The observed object points project on the image plane of the camera in the ini- 
tial and desired location onto the image points p 1 " 1 , ■ ■ ■ -,P % n % G K 3 (initial view) and 
pf es , . . . , p^ s G R 3 (desired view). These image points are expressed in homogeneous 
coordinates according to 

vT = I vti I , pf es = I i>t.2 I • r-> 

where p\ n ^ , pf\ s G K are the components on the x-axis of the image screen, while 
vTiiPt | s ^ R are those on the y-axis. The projections p\ nl andp des are determined by 
the projective law 

jini^ini ts Tyirii' ( n ±ini\ 

d des p des = KR des' ^. _ t des^ 

where d\ nl , d des G K are the depths of the zth point, qi G M 3 is the ith point expressed 
with respect to F abs , and K G M 3x3 is the upper triangular matrix containing the 
intrinsic parameters of the camera. 

The problem we consider in this paper consists of planning a trajectory from the 
initial location F vm to the desired one F des (which are unknown) by using the available 
estimates of: 

1 . the image projections p 1 ™ , pf es , . . . , p™ % , p^ ; 

2. and intrinsic parameters matrix K. 

This trajectory has to take into account physical constraints such as the limited field of 
view of the camera and the robot workspace, and has to optimize a desired performance. 
In the sequel, we will indicate with 5*0(3) the set of rotation matrices in R 3x3 , and 
with SE(3) the set of frames in the 3D space, where SE(3) = SO(3) x R 3 . 



2.2 Representation of Polynomials 

Before proceeding, let us briefly introduced a tool for representing polynomials which 
will be exploited in the sequel. Let p(x) be a polynomial of degree 2m in the variable 
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X = (*!,..., x n )' Gl",i.e. 

P(x)= Yl c ii,...,in x l 1 -" x n ( 4 ) 

i\ + . . . + i n < 2m 
ti>0,...,t„>0 

for some coefficients Ci x i n £ R. Then, p(a;) can be expressed as 

p(x) = x {m} ' P(a)x {m} (5) 

where x^ m ' is any vector containing a base for the polynomials of degree m in x, and 
hence can be simply chosen as the set of monomials of degree less than or equal to in 
in x, for example via 

X = (1, X\ 1 . . . , X n , X^,X\X2, ■ ■ • , X n ) , (O) 

and 

P(a) = P + L(a) (7) 

where P — P' is a symmetric matrix such that 

p(») =.T {m} >x {m} , (8) 

while L(a) is a linear parametrization of the linear space 

C(n,m) = {L = L' : x^' Lx^ = Vx} (9) 

being a a vector of free parameters. The dimension of x^ m ' is given by 

(n + m) ! 
n!rn! 
while the dimension of a (i.e., the dimension of C) is 



a(n,m) = ^^ (10) 



r(n, m) = — <j(n, m)(cr(n, m) + 1) — <r(n, 2m). (11) 

The representation in (O was introduced in [20| with the name SMR in order to investi- 
gate positivity of polynomials via convex optimizations. Indeed, p(x) is clearly positive 
if it is a sum of squares of polynomials, and this latter condition holds if and only if 
there exists a such that 

P{a) > (12) 

which is an LMI. It turns out that, establishing whether an LMI admits a feasible so- 
lution or not, amounts to solving a convex optimization problem. The matrices P and 
P{a) are known as SMR matrices of p(x), and can be computed via simple algorithms. 
See also |21]22|23J. 

3 Trajectory Planning 

This section describes the proposed approach. Specifically, we first introduce the parame- 
trization of the trajectory, we then describe its computation, and lastly we discuss the 
estimation of the initial camera pose and object points. 
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3.1 Trajectory Parametrization 

Let us start by parameterizing the trajectory of the camera from the initial location to the 
desired one. This can be done by denoting the frame of the camera along the trajectory 
as 

F(a) = {R(a),t(a)} (13) 

where a G [0, 1] is the normalized trajectory abscise, R(a) G 5*0(3) is the rotation 
matrix of F(a), and t(a) £ R 3 is the translation vector. We choose the convention 



a = -► F(a) = F 
a = 1 -> F(a) = F' 



III! 

des (14) 



The functions R : [0, 1] -> 50(3) and t : [0, 1] -> R 3 must satisfy the boundary 
conditions 

R(0) = R ini , R{1) = R des 
t(0) = t m \ i(l) = i 



(15) 



where R m \ R des , t ml and t des are the available estimates of R m \ R des , t mt and 
t des (the computation of these estimates will be addressed in Section [33t . We adopt 
polynomials in order to parameterize R(a) and t(a). Specifically, we parameterize t(a) 
according to 



t(a) = ^iV (16) 



i=0 

where 8 is an integer representing the chosen degree for t(a), and to, . . . ,is eK 3 are 
vectors to be determined. Then, we parameterize R(a) as 

E{r{a)) 

R ( a > = II I MI2 (17) 

where E : R 4 — > 50(3) is the parametrization of a rotation matrix via quaternions and 
r : [0, 1] G R 4 is the parameter along the trajectory. In particular, 

(r\ -r\ -r\ +rj 2 (nr 2 ~ ?' 3 r 4 ) 2(r 1 r 3 + r 2 r i ) \ 

B(r)= 2(nr 2 +r 3 r4) -r2 + r2-r|+r| 2 (r 2 r 3 - nr 4 ) . (18) 

\ 2(rir 3 -r 2 r 4 ) 2 (r 2 r 3 + nr 4 ) -rf - r| + r^ + r\ ) 

It turns out that 

E{r) G 50(3) Vr G R 4 \ {0 4 }, (19) 

and moreover 

VR G 50(3) 3£(£) G R 4 \ {0 4 } : B(f(i2)) = R- (20) 

A possible choice for £(R) is given by 



t(R) = 



I sin -m 

2 



(21) 



V 



cos - 

2 
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where 9 E [0, it] and u £ K 3 , ||u|| = 1, are respectively the rotation angle and axis in 
the exponential coordinates of R, i.e. 

R = e [eu] x. (22) 

We parameterize r(a) according to 

7 



r(a) = Y,ha l (23) 



4=0 



where fa, . . . , f 1 6R 4 are vectors for some integer 7. The boundary conditions in (fl~5l > 
become 



r- , . w (24) 



which imply that r(a) and t(a) can be re-parameterized as 

\ l na* + $(R tm ) 

(25) 



r(a) = ( £(R des ) - £(#"*) ~ Y2=i r i )a' + Y2=l ^ + £(#»*) 



t(a) = [ t des - t m% - YZi U) a d + Ei=i Ua l + **"* 

where fi, . . . , r 7 _i € K 4 and i\, . . . , £5-1 £ K 3 are free vectors. 

Let us observe that the derived parametrization can describe arbitrarily complicated 
trajectories, simply by selecting sufficiently large degrees 7 and 5. Moreover, it is useful 
to observe that special cases such as straight lines are simply recovered by the choices 

7 = 1 (straight line in the domain of E) 
5 = 1 (straight line in the translational space). 

For ease of description we will assume 7 = 1 in the following sections. 



3.2 Trajectory Computation 

In this section we address the problem of imposing constraints satisfaction and perfor- 
mance optimization on the parametrized trajectory via LMIs. Due to space limitation, 
we consider only two constraints, namely the visibility and the workspace constraints, 
and only one performance, namely minimizing the length of the trajectory. 

Let us consider first the visibility constraint, and let us indicate with Pi(a) = 
(Pi,i(a),Pi,2(a), 1)' £ R 3 the image projection of the ith object point along the tra- 
jectory. This point is visible whenever 

Pi,j(a) £ (si t i,s i>2 ) Vi = 1, . ..,nVj = l,2Vo € [0,1] (27) 

where si,i, S1.2, 82,1, ^2,2 £ K are the screen limits. We estimate Pi(a) via 

»«-OT + M*--CT5) + -(*--5ot) <28) 
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where /.(a) = (/i 1 i(o),/i I a(o),/ i ,3(o)) / 6 K 3 is 

f i {a) = kE{r{a))'{q i -t{a)) (29) 

and fa £ R 3 is the estimate of the object point qi (the computation of this estimate will 
be addressed in Section [331 l. Let us observe that this choice ensures Pi(0) = p" 11 and 
Pi(l) = pf es - We can rewrite Pi{a) as 

I /ft,i(°0\ 
Pi(a) = — -p- ft 2 (a) (30) 

A3(fl) \/w(«)/ 
where ft,i(a), ft ,2(0) S K are polynomials. Hence, visibility is ensured whenever 

%)>0V%)eWV(ie[0,l] (31) 

where 

w = {sj,fe/»,3(o) - (-l) fc ft.j(a) Vi = l,...,n, i,fc= 1,2} 

U{/ i>3 (a)Vz = l,...,n}. lJZj 

For each polynomial h(a) in 7i, we introduce an auxiliary polynomial Uh(a) of some 
degree, and we define 

v h (a) — h{a) - o(l - a)u h (a). (33) 

We express these polynomials via the SMR as 

Ufc(o) = y h (a)'U h y h (a) 
v h (a) = z h (a)'V h {a h )z h (a) 

where yh(a), Zh(a) are vectors containing polynomial bases, and [/, V(a/,) are sym- 
metric SMR matrices (see Section l2~!2l for details). 

Then, let us consider the workspace constraint. A possible way to define the 
workspace constraint is via inequalities such as 

(![ (t(a») —Oi)>Wi Vi = l,...,n w (35) 

where di £ K 3 is the direction along which the constraint is imposed, a,; £ [0, 1] spec- 
ifies where the constraint is imposed on the trajectory, Oi £ M 3 locates the constraint, 
Wi £ R specifies the minimum distance allowed from the point Oj along the direction 
di, and n w is the number of constraints. 

Lastly, let us consider the minimization of the length of the trajectory. A reasonable 
approximation that allows one to simply tackle this problem consists of minimizing any 
norm of the coefficients of the polynomial 

s(a) = t(o) - i ini - (i des ~ t mi ) a k (36) 

where k in any integer satisfying 1 < k < 8. Indeed, let us observe that the polynomial 

( lm + (i des — i lm ) a k describes a straight line from i tm to i des for any admissible 
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choice of k. Clearly, only the nonzero coefficients of s(a) need to be considered, and 
we denote the scalar entries of these coefficients with s\, . . . , s„ s (observe in fact that 
s(a) is a vector function). 

The sought trajectory can be obtained by solving the optimization problem 

' U h >0 Vft(o) £ H 
V h (a h ) > Vfc(o) 6 W 
minz s.t. ^ d£ (<(o») — Oj) > «;» Vi = 1, . . . ,n w (37) 

z > Si \/i = 1, . . . ,n s 
z > —Si Vi = 1, . . . ,n. s 

where z £ R is an additional variable. This problem belongs to the class of eigen- 
value problems and is a convex optimization problem with LMI constraints. Once the 
problem has been solved, one can obtain the 3D trajectory from | [25t . and the sought 
image trajectory from (f27l> — <f29b. Observe that z* = implies that the 3D trajectory is 
a straight line. 

3.3 Camera Pose and Scene Estimation 

In the previous sections we have described how the trajectory of the camera can be pa- 
rameterized and computed. In particular, the parametrization was based on the estimates 
R m% , R s , t m% and t des of the components of the initial and desired frames F %m and 
pdes ^ w jjji e me computation was based on the estimates q\ , . . . , q n of the object points 
qi, . . . ,q n . Here we describe some ways to obtain these estimates. 

Given the estimates p" il ,p des , . . . ,p™*,p^ es of the image projections and K of the 
intrinsic parameters matrix, one can estimate the camera pose between F m% and F des , 
and hence R ml and t %m since F des can be chosen without loss of generality equal to 
F abs . This estimation can be done, for example, through the essential matrix or through 
the homography matrix, see for instance [24 25 26 1 and references therein. 

Once that the estimates R m% and t im have been found, one can compute the estimates 
qi, . . . , q n of the object points via a standard triangulation scheme, which amounts to 
solving a linear least-squares problem. 

Let us observe that, if no additional information is available, the translation vec- 
tor and the object points can be estimated only up to a scale factor. In this case, the 
workspace constraint has to be imposed in a normalized space. This problem does not 
exist if a CAD model of the object (or part of it) is available, since this allows to estimate 
the distance between the origins of F vm and F des . 



4 Illustrative Example 

In this section we present an illustrative example of the proposed approach. Let us 
consider the situation shown in FigureQJ, where a camera observes some object points 
(the centers of the nine large dots in the "2", "3" and "4" faces of the three dices) 
from the initial and desired locations (leftmost and rightmost cameras respectively). 
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Fig. 1. (a) Initial frame F ml (leftmost camera), desire frame F des (rightmost camera), object 
points (centers of the nine large dots in the "2", "3" and "4" faces of the three dices), planned 
trajectory (solid line), and some intermediate locations of the camera along the planned trajectory, 
(b) Image projections of the object points in the initial view ("o" marks) and desired view ("x" 
marks), and image projection of the planned trajectory (solid line). 
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Fig. 2. Results obtained by planning the trajectory in the presence of image noise and calibration 
errors, and by tracking the planned trajectory with an image-based controller 
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Fig. 3. Camera coordinates during the motion shown in Figure|2] 
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Figure QJ) shows the image projections of these points in the initial view ("o" marks) 
and desired view ("x" marks). The intrinsic parameters matrix is chosen as 

'400 320 N 
K = \ 300 240 | . (38) 

1 

The problem consists of planning a trajectory from the initial location to the desired one 
which ensures that the object points are kept inside the field of view of the camera and 
the camera does not collide with the sphere interposed between F m% and F des (which 
represents an obstacle to avoid). It is supposed that the position of the obstacle is known 
with respect to F des , and that the distance between two points of the dices is known. 

Let us use the proposed approach. We parameterize the trajectory as described in 
Section 13.11 with polynomials of degree two by estimating the camera pose between 
pmi an( j pdes v j a me essen ti a i matrix. We build the set of polynomials H, which im- 
pose the visibility constraint, and we compute the SMR matrices Uh and Vh{cih) in 
d34l . The workspace constraint is chosen by requiring that the trajectory must remain at 
a certain distance from the obstacle in three directions. Then, we solve the eigenvalue 
problem (|37] |. and we build the 3D trajectory from d25l l and the image trajectory from 
d27])-(l29t. which are shown in FiguresQJ andQJt. 

Now, in order to introduce typical uncertainties of real experiments, we corrupt the 
image projections of the object points by adding image noise with uniform distribution 
in [— 1, 1] pixels to each component. Moreover, we suppose that the camera is coarsely 
calibrated, in particular the available estimate of the intrinsic parameters matrix is 

'430 338 N 

K = \ 275 250 | . (39) 

1 

We repeat the previous steps in the presence of these uncertainties, and then we track 
the planned trajectory by using the image-based controller proposed in [ 13 1. Figures^ 
and[2j3 show the obtained results: as we can see, the camera reaches the desired location 
by avoiding collisions with the obstacle in spite of the introduced uncertainties. Figures 
[3^ and[3t> show the camera coordinates during the motion. 



5 Conclusions 

This paper has proposed an approach for planning image trajectories in visual servo- 
ing based on the use of polynomials and convex optimization problems. It has been 
shown how constraints satisfaction and performance optimization can be imposed on 
the sought trajectory through LMIs by adopting the SMR. The planned image trajec- 
tory is then tracked by using an image-based visual servoing controller. 
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Abstract. This paper introduces an originally designed tracked robot. This robot 
belongs to the VGSTV (Variable Geometry Single Tracked Vehicle) category, 
which is actually a sub-group of Variable Geometry Vehicles well represented by 
the iRobot Packbot. Those robots have been used several times for search and 
rescue mission and seems to be a real asset because of their clearing capability. 
After a brief categorization, technical specification of our robot are presented and 
geometrical and dynamical models are computed in order to compare static and 
dynamic balance. Then, thanks to the results of the balance study, an autonomous 
staircase clearing controller based on artificial neural network is computed and 
tested. A general conclusion about possible improvements and future work ends 
the paper. 



1 Introduction 

UNMANNED GROUNDED VEHICLE (UGV) is a topically research field applied to 
a wide range of applications like for example exploration or missions in hostile envi- 
ronments. Research laboratories and robotics companies are currently working on the 
design of tele-operated and autonomous robots. According to [3 1 and [ 2 1 UGVs can be 
classified into three categories: Man-packable, Man-portable and Not man-portable. 

The robots presented in this paper are classified in the man-packable and man- 
portable categories. In this class of robots, designers have to face the following 
dilemma: on one hand, build a small robot that can be easily carried and move into 
narrow environments. Unfortunately, it will generally result in poor obstacle clearing 
capability. On the other hand, build a bigger robot will increase its ability to surmount 
obstacles but will not enable the robot to go through narrow openings. The challenge 
is then to build a robot as small as possible with the higher obstacle clearing capa- 
bility. Based on this observation the first part of this paper introduces the existing ex- 
perimental and commercial robots and discusses about their clearing capabilities. The 
following of the paper describes an originally designed UGV (Fig. |3J. This robot can 
be classified into the Variable Geometry Single Tracked Vehicle (VGSTV) category, 
i.e. it has the mechanical ability to modify its own shape according to the ground con- 
figuration. The design of our prototype is described in the third part with a short dis- 
cussion about the technical choices (information can be found on the project website: 
http://www.istia.univ-angers.fr/LISA/B2P2/b2p2.html). The next section introduces the 
dynamic model of the robot. The first tele-operated experiments have shown that the 
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mass distribution is crucial to pass through large obstacles. Then, thanks to the results 
of the balance study, an autonomous staircase clearing controller based on artificial neu- 
ral network is computed and tested. A general conclusion about possible improvements 
and future work ends the paper. 

2 Existing UGVs 

2.1 Wheeled and Tracked Vehicles with Fixed Shape 

This category gathers non variable geometry robots (fig. QJ and b). Theoretically, this 
kind of vehicles are able to climb a maximum step twice less high than their wheel 
diameter. Therefore their dimensions are quite important to ensure a large clearing ca- 
pability. This conception probably presents a high reliability [ 1 1 but those robots cannot 
be easily used in unstructured environments like after an earthquake 0. 



■?m?" 




Fig. 1. a): Talon-Hazmat robot (Manufacturer: Foster-Miller), b): ATRV-Jr robot. Photo Courtesy 
of AASS, rebro University c): Packbot (manufacturer: IRobot), d): RobuROC 6 (Manufacturer: 
Robosoft), e): Helios VII. 



2.2 Variable Geometry Vehicle 

A solution to ensure a large clearing capability and to reduce the dimensions consists in 
developing tracked vehicles which are able to modify their geometry in order to move 
their center of mass and climb higher obstacles than their wheel's diameters. 

The Packbot robot (Fig.QJ) is probably one of the most famous commercial VGTV 
(Variable Geometry Tracked Vehicle). This robot is equipped with tracks and two ac- 
tuated tracked flippers (372 mm long). The flippers are used to step over the obstacles. 
The obstacle clearing capability of this kind of VGTV depends on the size of the flip- 
pers. For more information and a detailed survey on clearance capability of the Packbot 
the reader can consult [6|. 

The robuROC6 (Fig.QJl) is equipped with 46.8 cm diameters wheels and can clear 
steps until 25cm (more than half the diameter of the wheels). Joints between the axles 
make this performance possible. An other original system called Helios VII (FigQJ) |7 1 
is equipped with an arm ended by a passive wheel which is able to elevate the chassis 
along a curb. 



2.3 Variable Geometry Single Tracked Robots 

Actually, there is a subgroup in VGTV called Variable Geometry Single-Tracked Vehi- 
cles (VGSTV) jT3l . It gathers robots equiped with as tracks as propulsion motors. In 
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Fig. 2. a): Micro VGTV (manufacturer: Inuktun Ltd) b): VGSTV mechanism c): Viper robot 
(Manufacturer: Galileo) d): Rescue mobile track WORMY 



most cases those robots are equipped with one or two tracks (one for each side). It can 
be divided into two groups: 

- robots with deformable tracks, 

- robots with non deformable tracks. 

Non-deformable Tracks VGSTV. The most famous example is the Micro VGTV. 
Illustrations of a prototype manufactured by the company Inuktun are presented on Fig. 
[2^. This robot is based on an actuated chassis used to modify the shape of the robot. 
The right picture of Fig. [2^ shows the superimposing configurations. The tracks are kept 
tightened by a passive mechanism. The robot is thus equipped with three motors: two 
for the propulsion and one for the chassis joint. 

Non commercial vehicles exists in the literature as the VGSTV mechanism (Fig.[2j3) 
which is dedicated to staircase clearing. It is composed of two tracks and two artic- 
ulations which allow it to have many symmetrical configurations such as a rectangle, 
trapezoids, inverse trapezoids etc. 

Many other VGTV architecture exist, for further information reader can consult |[T8ll . 

oa, a, q and na. 

Deformable Tracks VGSTV. Some single tracked robots have the ability to modify 
the flexing of their tracks. Two examples presented on Fig.[2]c and d, are able to adapt 
their shape to obstacles [ 1 2 1 . However, even if the control of the robot seems easier with 
a flexible track than with a non flexible one, the mechanical conception could be more 
complicated. 

According to the presented state of the art, for general purpose missions we beleive 
that the best compromise between design complexity, reliability, cost and clearing ca- 
pabilities is the Variable geometry single tracked robots category. The next section will 
introduce and describe our prototype of VGSTV. 



3 Prototype Description 

The main interest of VGSTV (equipped with deformable tracks or not) is that it is prac- 
ticable to overcome unexpected obstacles lfT"3l . Indeed, thanks to the elastic property of 
the tracks the clearance of a rock in rough terrain will be more smoothly with a VGSTV 
(e. g. FigO than with a VGTV (Fig.Q]c, d and e). On the Micro VGTV presented on 
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Fig. 3. B2P2: Clearing of a curb 



Fig. [2 the tension of the tracks is mechanically linked with the chassis joint so it is 
constant during the movement. Nevertheless, in some cases, less tense tracks could in- 
crease the clearance capability by increasing the adherence. An interesting study about 
this point was developed by [8| giving a VGSTV able to climb staircases where the 
tension of the tracks was mechanically managed as on the Micro VGTV (Fig.|2^). How- 
ever, this system was equipped with a spring to allow the tracks to adapt their shape to 
the ground (depending on the strength of the spring). 

The conception of our prototype is based on this previous work, but we decided 
to actuate the tension of the tracks. Indeed, by using two motors instead of one (Fig. 
[4]> it is possible to increase the tracks adaptation to the ground developed by |8| and 
reach new configurations for the robot. As example, the solution proposed in this paper 
allows our robot to adopt classical postures of VGSTV (Fig. Oa), \3\b) and^c) ), but 
also other interesting positions. On Fig.[3]B2P2 is clearing a curb of 30 cm height with 
tense tracks. The position of the robot on Fig [3c) can also be obtained with the Micro 
VGTV, but it is a non-safety position and B2P2 is close to topple over. On Fig. [2d) the 
tracks have just been released. They take the shape of the curb and it can be cleared 
safely. This last configuration outlines the interest of using an active system instead of 
a passive one. Consequently, although our prototype (Fig. [2j>) belong to the VGSTV 
category and have not deformable tracks, it has the ability to adapt them to the ground 
(as deformable ones). 

Besides, even if there is a risk of the tracks coming off, loosening the tracks may be 
an efficient mean of increase the surface in contact with the floor in rough terrain and 
then to improve the clearing capability of the structure. By the way, the risk could 
decreased by using sensor based systems to control the tension of the tracks or by 
modifying the mechanical structure of the robot (adding some kind of cramps on the 
tracks or using a guide to get back the tracks before it comes off). 



3.1 Mechanical Description 

This UGV is equipped with four motors. Fig.[4]presents the integration of the motors in 
the robot. Motors 1 and 2 are dedicated to the propulsion (tracks). 
The actuated front part is composed of motors 3 and 4: 

- Motor 3 actuates the rotational joint, it allows the rotation of the front part around 
the second axle. 

- Motor 4 actuates a driving screw, it controls the distance between the second axle 
and the third one. 

To keep the tension of the tracks the trajectory of the third axle is given by an ellipse 
defined by two seats located on the first and the second axle. 
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Fig. 4. Overview of the mechanical structure, side and top view of the real robot 



L+(L 2 + L 3 ) = K (1) 

where the lengths L, L-i and L3 are referenced on Fig. [4] K is a constant parameter 
depending on the length of the tracks, L3 evolves in order to achieve equality (fl]l and is 
linked to the angle 6 in the following manner: 



L, 



L\ - K 2 



2(Licos(tt- I 0\)-K) 



u 



(2) 



4 Dynamic Model 

This section deals with the dynamic model of the robot which is based on the geometric 
model (Fig.0 detailed on ifTTl . 

According to this model, the robot motion in a 3D frame (Ro) is described by the 
vector q of the 8 joints variables: 

q = [qi,q2,q3,qA,q5,q6,qT,q8] T 

The dynamic model of a mechanical system establishes a relation between the effort 
applied on the system and its coordinates, generalized speeds and accelerations ([5| 
and ifTOll ). In this section, the following notations are used : 

- j describes the joints from 1 to 8, 

- i describes the segments from 1 to 3 (referenced on FiglUl, 

- n and m describes indexes from 1 to 8. 



4.1 The Dynamic Equations 

The general dynamic equations of a mechanical system is: 

d dL dL 



dt dqj dq 3 



= Q J +T J 



(3) 



178 J.-L. Paillat, P. Lucidarme, and L. Hardouin 




Fig. 5. B2P2's geometric model. Joints 1,2 and 3 represents the robot position. Joints 4, 5 and 6 
symbolize respectively the yaw, roll and pitch, 7 and 8 are the actuated joints. 



- L is the Lagrangien of the system. It is composed of rigid segments, so there is no 
potential energy. Although the Lagrangien corresponds to the kinetic energy. 

- qj is the j joint variable of the system. 

- Qj is the gravity's torque applied to the j fh joint of the system. 

- Tj is the external force's torque applied to the j th joint of the system. 



The kinetic energy is given by: 



1 1 



(4) 



- wii is the mass of the i th element of the model, 

- Vi is the linear speed of the i th element's center of gravity, 

- Wi is the angular speed of the i th element's center of gravity, 

- Ii is the matrix of inertia of the i th element of the system. 

In order to have homogeneous equations, Wi is defined in the same frame as Ii ; it allows 
to formulate Vi and Wi according to q: 



Vi = J Vi (q)q 



(5) 



Wi = RojJwi(q)q 



(6) 



where J Vi and J Wi are two matrices and Rqj is the transport matrix between the frame 
i?o and the frame j linked to the segment i. 
The kinetic energy formula is: 

K = ^X^-Mfff-^fo) + JlM^hRl^MM (7) 



which can be rewritten as: 



K = -q T D(q)q 



(8) 
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by developing the previous formula, we obtain: 

K = - ^2 d m ,n{q)q m q n (9) 

where d mtTt (q) is the m, n th element of the matrix D{q). 
The gravity's torque is given by: 

- G° zi is the z coordinate of the CoG of the i th segment's computed in the base frame 

(Ro), 

- g is the gravity acceleration. 

Vector T (defined in <£3l>) is composed of the external forces' torque. For the robot pre- 
sented here, there is no consideration of external forces, so the T vector only describes 
the motorized torques. Joints 1, 4, 7 and 8 are motorized, so the vector T is given by 
those four parameters. T\ and T4 are computed from the torques of motors 1 and 2 
while T-j and T% are deduced from motors 3 and 4. 
The Euler-Lagrange equations can be written as: 

^ d jm (q)q m + 2J c nmj (q)q n q m = Qj + Tj (11) 

m n,m 

c nm3 = \[^ + ^-^\ (12) 

2 dq n dq m dqj 

which is classically written as: 

D(q)q + C(q,q)q = Q + T (13) 

where D(q) represents the matrix of inertia and C(q, q) the centrifuge-coriolis matrix 
where Xj m , the jm th element of this matrix, is defined as: 

^jm — / J C-nmjqn- 
n 

Finally, the J V i and J w i matrix considered in Q and (0 have to be computed. 

4.2 J v i and J w i Matrix Formulation 

The matrix which links articular speed and general speed of a segment is computed from 
the linear and angular speeds formulas. The goal is to find a matrix for each segment. 
They are composed of 8 vectors (one for each joint of the model). 

The computation consists in formulating in the base frame, the speed (Vp i (j — 
1) j) R °) °f a point Pi given by a motion of the joint qj attached to the frame j accord- 
ing to the frame j — 1. Those parameters can be deduced from the law of composition 
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speeds and the Denavitt Hartenberg (DH) formalism used for the geometric model IfTTl . 
Indeed, the general formulation is simplified by the geometric model. Only one degree 
of freedom (DoF) links two frames using the DH model and this DoF is a revolute or 
a prismatic joint. Moreover, the Z axis is always the rotation or translation axis, so the 
angular and linear speeds are given by four cases: 



- The angular speed of a point for a revolute joint: 

w P {j-l,j) Ro = Ro,j 

- The linear speed of a point for a revolute joint: 



<Lr 



vp(j - 1, j) Ro = V^- 1 + Vp 3 + Wj A OjP R > 

"0" 



ARojPj- 



- The angular speed of a point for a prismatic joint: 

wp(j-l,j) = 

- The linear speed of a point for a prismatic joint: 

v P (j-l,j) R °=R ,. 



Qj 



(14) 



(15) 



(16) 



(17) 



where Pj is the P point's coordinates in Rj. 

Thus, the matrix of a segment i is formulated by computing speeds for each joints 

as: 



= J(q)4 = [Ji,»(g), J2,%{<i), -J&Ai)} i 



(18) 



iii 



where Jji(q) is a vector which links the speed of the i segment according to the j 
joint. The first segment is not affected by the motion of joints 7 and 8 while the second 
is not affected by joint 8, therefore Ji^{q), J% : \(q) and Js,2{q) are represented by a 
null vector. 



5 Balance Criterion 

The balance criterion used here are the ZMP (Zero Moment Point), widely used for 
the stability of humanoid robots and the Center of Gravity (CoG). Previous theoretical 
works and experiments have proved the ZMP efficiency [19]. It consists in keeping the 
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point on the ground at which the moment generated by the reaction forces has no com- 
ponent around x and y axis ( 1111 and |9j) in the support polygon of the robot. When the 
ZMP is at the border of the support polygon the robot is teetering. Unlike the ground 
projection of the center of gravity, it takes into account the robot's inertia. 

The purpose of the following is to defined the coordinates of this point in any frame 
of the model according to the configuration of the robot. The definition can be im- 
plemented into the Newton equations to obtain those coordinates. In any point of the 
model: Mo = M z + OZ A R {Mq and M z define respectively the moment generated 
by the reaction force R at the points and z). 

According to the previous definition, there is no moment generated by reaction forces 
at the Zero Moment Point. Consequently, if Z defines the ZMP coordinates Mq = 
OZ A R. This formulation can be implemented into the Newton equations as: 

5 a = Mo + OGAP + OGAFi (19) 

where P is the gravity force, G is the robot's center of gravity and Fi is the inertial 
force (the first Newton's law gives Fi — —mG). According to the ZMP definition, the 
equation ( IT9b can be formulated as: 

S a = OZ AR + OGAP + OGAFi (20) 

(21) 



$0x — ZyR-z + G y P z —G z P y — G z Fi y 



Soy — ~Z X R Z + G Z P X — G X P Z 

I z y - R z 

S r, -6 n „+G r P„-G„P, 



(22) 



Also, it is possible to compute the position of the ZMP as a function of q (<5 depends 
on the matrix D(q)). 

Assuming the ground knowledge, the ZMP computation gives a criterion to determi- 
nate the stability of the platform. 

6 Results 

This section presents the numerical computation of the criterion in the case of the clear- 
ance of a staircase (staircase set of 15 cm risers and 28 cm runs) with an average speed 
of 0.13 m.s^ 1 . The robot is equipped with a 2-axis inclination sensor that provides 
rolling and pitching. Vector q entries are measured using encoders on each actuated 
axis of the robot. Data have been stored during the experiments and the models (CoG 
and ZMP) have been computed off-line. This computation does not take into account 
the tracks' weight which is negligible in regard with the robot's weight. Fig.|6]presents 
the evolution of the ZMP (left) and the difference between those two criterion (right) 
during all the clearance. Pi P2 and P3 represents the z-coordinates in the frame R5 
(Fig. [5]) of three points of the robot which localization are noticed on Fig. [4] 

This experiment allows us to validate the presented model and confirms the compu- 
tation of the ZMP criterion. However, as it is shown on Fig. [6] the average difference 
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Fig. 6. Experiment's results. The left chart represents the evolution of the ZMP and the right one, 
the difference between CoG and ZMP. 

between the ZMP and the COG is insignificant (about 0.21 %). Moreover, the two peaks 
(A and B) on the Fig.[6]are not due to the dynamics of the system but to measurement 
errors. As the acceleration is measured with the encoders (linked to the motor shaft), 
when the tracks slip, the measurement is erroneous. The ZMP is computationally more 
expensive, needs more sensor measurements and the difference with the CoG is negli- 
gible. For these reasons, we conclude that the CoG seems well suited for this kind of 
experiments. Anyway, in the case where fast obstacle clearance may be necessary, the 
CoG may not longer be considered and the ZMP must be used instead. 

7 Autonomous Stair Climbing Controller 

The purpose of the following is to present a reactive autonomous staircase climbing 
controller. This study was performed through a home made C++ software which simu- 
lates the behavior of a VGTV in staircase climbing situations. The simulator is based on 
the model of our prototype B2P2. Giving the reduced impact of dynamics effects on the 
robot as explained previously, the CoG is considered as balance criterion in the simu- 
lations. The system presented here consists in controlling the elevation of the front part 
(e.g. the shape of the robot and implicitly the position of CoG). Currently, the tension of 
the tracks is controlled by a man-programmed algorithm that automatically adapts the 
shape of the robot according to ground. 

7.1 Entries of the System 

The system has to be able to react differently in regards with the climbing stage. So, 
entries have to differ according to the stage (first step, middle steps, final step). The 
chassis inclination measurement allows to check out the first stage (if the ground is 
plane) but does not help to conclude about the two others stages (Fig. [7]). On the other 
hand, a distance sensor could check out the last stage as it is shown on Fig. [JJ So 
considering that the robot is always parallel to the step in front of it, the inclination and 
distance sensors could be sufficient to achieve an autonomous staircase climbing. 



7.2 Controller 

As explained by |[T6l genetic algorithms can be used to train and optimize control sys- 
tems. Typically, Artificial Neural Networks (ANNs) are often used as control systems 
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Fig. 7. Stage decomposition during climbing 

for obstacle avoidance or grasping tasks ([ 14 J) because such controller can approximate 
a wide range of mathematical functions. An ANN is composed of several units linked by 
weighted connections u)y . Each unit i has entries Xj and one output y,; which is a func- 
tion er() of J2j ( w ij - x j) where Wij corresponds to the weight of the link between two 
neurones. Theoretically, neurones are organized in layers to perform a neural network. 
This network can be feed-forward (signal travel from inputs units forward to output 
units) or recurrent (there may be feedback connections from neuron in upper layer or 
in the same layer). As introduced, in the case of autonomous staircase climbing with a 
VGSTV as B2P2, the system have to control the elevation of the front part according 
to the position of the robot in the staircase. Otherwise, the output of the network is the 
elevation of the front part. Consequently the neural network architecture chosen is a 
feed-forward network with one hidden layer (Fig.[8ll that must be addressed to approxi- 
mate non linear functions. Indeed, a recurrent network does not seem useful, because a 
variation of the output (elevation angle) includes a variation of the network entries (IR 
distance sensors and inclination sensor). 

7.3 Evolutionnary Training 

In the controller, all the parameters are known except the 15 synaptic weights (wi) 
which are deduced by an evolutionary algorithm based on a classical genetic approach. 
As the structure of the network has been fixed, only those weights have to be optimized. 
Consequently, a chromosome is only composed of the weights Wi. 

Table 1. Chromosome description 



Wl 


W2 




W, 




W14 


w 15 



Selective Reproduction. Each generation is composed of 200 sets of chromosomes 
composed of fifteen parameters randomly selected into [—1,1]. After each generation, 
a selective reproduction is performed in order to compute the next generation. Here this 



184 J.-L. Paillat, P. Lucidarme, and L. Hardouin 



o(x) - tanh(kx) 




Chassis inclination 
(-90;+90)) 



Elevation angle 
(-90;+90) 



Fig. 8. Neural Network Model 

selection is roulette wheel based that allow the best individual to be statistically selected 
more frequently. Otherwise: 

fi 

Where pi is the probability of selection for the individual i, fi stands for the fitness of 
the individual i and N represents the number of individual in the previous generation. 

A selection process is performed for each chromosome element (e. g. fifteen times) 
to compute the next generation. However, the best individual of each generation is du- 
plicated for the next generation without selective reproduction. Additionally, a mutation 
process is performed by replacing by a random value in the range [—1,1] a randomly se- 
lected gene (chromosome element) for 10% of the new generation to prevent premature 
convergence. 



Fitness Function. As the goal is to climb staircases, the fitness function must be linked 
to the number of steps cleared. As this parameter is discrete, we combined it with an- 
other parameter that minimizes the energy (directly linked to the elevation part move- 
ment). Moreover, minimizing the energy provides smooth trajectory of the robot. It is 
expressed as follow: 



fi 



N, 



steps 



E 



Where /,; is the fitness of the individual i, N st e P s stands for the number of steps cleared 
by individual i and E represents parameter depending on the average speed of the front 
part (V) as: 

' 1 if V < threshold 
V otherwise 



E 



7.4 Results 

The simulation was performed with a classical staircase sets of five steps of 15 cm 
risers and 25 cm runs. Fifty generations was tested in order to have a characteristic 
convergence (Fig. |9(a)) . Simulation results are shown on Fig. |9(b)| 
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tion results ble 

Fig. 9. Results 

In order to evaluate how generic are the produced results, the best individual has to 
get over several kind of staircases as described in table |9(c)] The percentage of cleared 
steps is indicated according to the length and the height of the steps. Note that the 25 cm 
is the maximum step height that the prototype is able to clear in tele-operated staircase 
clearance. 

The controller seems to be able to perform an autonomous staircase clearing in regard 
with the quality of information given by the environment. 

8 Conclusions 

This paper presents a way to climb staircases autonomously with classical VGTVs by 
using their ability to modify their geometry in order to adapt themselves to the ground. 
It consists in a neural network which compute the elevation of the robot's front part 
in respect to its inclination and the distance between it and the next step. However, 
it was necessary to study the behavior of our robot in order to validate robot's model 
in context of a staircase clearance. Those simulation results have been implemented 
on our prototype B2P2 ; A video of autonomous staircase climbing can be found at 
http://www.istia.univ-angers.fr/LISA/B2P2/b2p2.html. Some improvement should be 
done to increase the reliability but those first result seems promising. Our future works 
focuses on a survey about control of VGSTV equipped with n-degrees of freedom using 
interval analysis in order to maximize the adaptability of such robots. 
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Abstract. In this paper, we present an approach for passivity-based bipedal robots 
to achieve stable dynamic walking on uneven terrain. A powered two-dimensional 
seven-link walking model with flat feet and compliant ankles has been proposed to 
analyze and simulate the walking dynamics. We further describe a Particle Swarm 
Optimization based method, which uses optimized hip actuation and ankle com- 
pliance as control parameters of bipedal walking. Satisfactory results of simula- 
tions and real robot experiments show that the passivity-based walker can achieve 
stable bipedal walking with larger ground disturbance by the proposed method in 
view of stability and efficiency. 

Keywords: Passive dynamic walking, Bipedal robots, Uneven terrains, Stability, 
Modeling. 



1 Introduction 

Stability guaranteed dynamic bipedal walking is one of the keys but also one of the more 
challenging components of humanoid robot design. Several actively controlled bipedal 
robots are able to deal with such dynamic stability when walk on irregular surface [ 22 1 , 
step over obstacles [6] and climb stairs [ 1 1 1. However, to increase the autonomy of the 
robot, the locomotion efficiency, which is far from that of human motion, has to be 
improved. 

As one of the possible explanations for the efficiency of the human gait, passive 
dynamic walking 1 10 1 showed that a mechanism with two legs can be constructed so 
as to descend a gentle slope with no actuation and no active control. Several studies 
reported that these kinds of walking machines work with reasonable stability over a 
range of slopes (e.g. [ 10 1 , [ 1 1) and on level ground with kinds of actuation added (e.g. 
01, H2) show a remarkable resemblance to the human gait. In spite of having high 
energetic efficiency, passivity-based walkers have limits to achieve adaptive locomotion 
on rough terrain, which is one of the most important advantages of the legged robots. 
In addition, these walkers are sensitive to the initial conditions of walking. 

To overcome such disadvantages, several studies proposed quasi-passive dynamic 
walking methods, which implement simple control schemes and actuators supplemen- 
tarily to handle ground disturbances. For example, |[T3l described a reinforcement learn- 
ing based controller and showed that the walker with such controller can maximally 
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overcome —10mm ground disturbance. [15| presented a robot with a minimal num- 
ber of degrees of freedom which is still capable of stable dynamic walking even on 
level ground and even up a small slope. ifTTl designed a learning controller for a two- 
dimensional biped model with two rigid legs and curved feet to walk on uneven surface 
that monotonically increases from 12mm to 40mm with a 2mm interval. In these stud- 
ies, passivity-based walkers are often modeled with point feet or round feet. And the 
control parameter only includes hip actuation. None of them analyzed the stability or 
adaptability in quasi-passive dynamic walking with flat feet and ankle joint compliance 
which is more close to human motion. In fact, flat feet can offer the advantage of dis- 
tributing the energy loss per step over two collisions, at the heel and at the toe Q, lfT2ll . 
Moreover, experiments on human subjects and robot prototypes revealed that the tendon 
of the muscle in ankle joint is one mechanism that favors locomotor economy Q , lfl8ll . 
It is predictable that by controlling and optimizing the hip actuation and ankle compli- 
ance, the passivity-based bipedal walker may achieve adaptive bipedal locomotion with 
larger disturbance on uneven terrain. 

In this paper, we investigate how to control passivity -based bipedal walkers to achieve 
stable walking with terrain adaptability. A powered two-dimensional seven-link walking 
model with flat feet and ankle compliance has been proposed to analyze and simulate the 
walking dynamics. We hypothesized that the nervous system that controls human loco- 
motion may use optimized hip actuation and joint compliance to achieve stable bipedal 
walking on irregular terrain. Actually, hip actuation will cause the walker to change 
locomotive patterns. Furthermore, we use both simulations and real robot prototype to 
explore the performance of the passivity-based walkers when walk on uneven terrain 
by utilizing a biologically inspired optimization based controller, which is adaptive and 
capable of selecting proper hip actuation and ankle compliance in view of walking sta- 
bility and efficiency. 

This paper is organized as follows. Section 2 describes the walking dynamics of the 
biped with flat feet and ankle compliance. Section 3 presents the optimization based 
walking control method. Experimental results are shown in section 4. We conclude in 
Section 5. 

2 Passivity-Based Bipedal Walker 

Our model extended the Simplest Walking Model |5| with the addition of hip joint ac- 
tuation, upper body, flat feet and linear spring based compliance on ankle joints, aiming 
for adaptive bipedal locomotion with optimization based controller. The model consists 
of an upper body (point mass added stick) that rotates around the hip joint, a point mass 
representing the pelvis, two legs with knee joints and ankle joints, and two mass added 
flat feet (see Fig.Q}. 

The mass of each leg is simplified as point masses added on the Center of Mass 
(CoM) of the shank and the thigh respectively. Similar to lT9ll . a kinematic coupling 
has been used in the proposed model to keep the body midway between the two legs. In 
addition, our model added compliance in ankle joints. Specifically, the ankle joints are 
modeled as passive joints that are constrained by linear springs. The model is imple- 
mented in MATLAB, using the parameter values shown in Table [T] which are derived 
from the mechanical prototype. 
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Fig. 1. Two-dimensional seven-link passive dynamic walking model. Mechanical energy con- 
sumption of level ground walking is compensated by applying a hip torque. The global coordi- 
nates of the hip joint is notated as (xh, yn). <X\ and oli are the angles between each leg and the 
vertical axis in sagittal plane respectively. The knee joints and ankle joints are all passive joints. 
To simulate human ankle compliance, the ankle joints of the model are constrained by linear 
springs. 

The passive walker travels forward on level ground. The stance leg keeps contact 
with the ground while the swing leg pivots about the constraint hip. To compensate 
the mechanical energy consumption, similar to [8], we added a hip torque P between 
the swing leg and the stance leg. When the flat foot strikes the ground, there are two 
impulses, "heel-strike" and "foot-strike", representative of the initial impact of the heel 
and the following impact as the whole foot contacts the ground. 

The walking model can be defined by the rectangular coordinates x which can be de- 
scribed by the x-coordinates and y-coordinates of the mass points and the corresponding 
angles. The walker can also be described by the generalized coordinates q. The springs 
on the ankles constrain the foot vertical to the shank when no heel-strike or foot-strike 
has occurred. 



2.1 Constraint Equations 

The constraint equations £i and £2 used to detect heel contact with ground are defined 
as follows: 



6 



Xh + li sin ai —Ifi cos a\ — Xf\ 
yh — h cos a\ — Ifi sin ai 

x h + l 2 sin a 2 — lf2 cos a 2 — 272 
Vh - h cos a 2 - lf2 sin a 2 



(1) 
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Table 1. Default Parameter Values for the dynamic walking model and the following simulations 



Parameter 


Description 


Value 


mi, rri2 


leg mass 


1.12% 


rri3 


upper body mass 


0.81% 


mi 


hip mass 


15.03% 


m lt , m 2t 


thigh mass 


0.56% 


mis, m,2s 


shank mass 


0.56% 


771/1, m/2 


foot mass 


2.05% 


luk 


leg length 


0.7m 


hi, 1'22 


shank length 


0.35m 


lfl,lf2 


foot length 


0.15m 


lb 


upper body length 


0.5m 


I w 


body width 


0.15m 


Cb 


CoM of upper bod) 


'0.2m 


Cl, C 2 


CoM of leg 


0.2m 


Cll,C 2 2 


CoM of shank 


0.2m 


k 


ankle stiffness 


8.65Nm/rad 


P 


hip torque 


0.38Nm 



where x/i and Xfi are the global x-coordinates of the latest strike. When the flat foot 
completely contacts the ground, the constraint equations £3 and £4 used to maintain foot 
contact with ground are defined as follows: 



6 = 

£4 = 



Xh + l\ sinai — Xf\ 
y h - ^cosai 

Xh + ^sina 2 - £/2 
Vh ~ h cos a 2 



(2) 



If only the heel contacts the ground, the constraint equations £5 and ^6 during the period 
before foot-strike are defined as follows: 



£5 = (xh + h sinai - x f i) 2 + (y h - h cosai) 2 
63 = {xh + h sin a 2 - x 2 ) 2 + (yh - h cos a 2 ) 2 - 



I 2 

;2 

'/2 



(3) 



Note that £5 and £g are similar to constraining the ankle joint that connects shank and 
foot to move in a circular orbit with heel as the center and distance between heel and 
ankle joint as the radius. 

Similar to 1 191, a reduced mass matrix M r is introduced, which is defined as follows: 



\M r 



[TV [M g ] [T] 



(4) 



clx 



where the Jacobian T = ^p. Here x is the global coordinates of the six pointmasses 
(stance shank with foot, swing shank with foot, hip, stance thigh, swing thigh, body), 
while M g is the mass matrix in global coordinates. Matrix Sj transfers the independent 
generalized coordinates q into the constraint equation £j, where i — 1, 2, . . . , 6 
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(5) 



dq 



Consequently, matrix H, is defined as follows: 

& _ d(Sjq) . 



(6) 



-M r ~f 
E 3 _ 




9 


= 


. — "3. 



2.2 Single Support Phase 

Suppose that leg 1 (l\) is the stance leg, while leg 2 (Z2) is the swing leg. In the beginning 
of the single support phase, the knee joint is locked (keep the shank and the thigh in a 
straight line). The Equation of Motion (EoM) is described as follows: 



(7) 



where F r is the external force, while F c is the foot contact force. Here, the external 
force F r is used to compensate the mechanical energy consumption of level ground 
walking, which defined as follows: 

{F r } = [T] T {{F}~[M g ]{i}) 

where F is the external force in global coordinates, including gravity, hip actuation, and 
torque in compliant ankle joints. Then when the swing leg is swung forward, the knee 
joint releases the shank. 



2.3 Heel-Strike Phase 

In this phase, leg 1 (l\) is still the stance leg, while leg 2 Q2) is the swing leg. The heel 
contacts the ground (heel-strike occurs). The EoM of the model changes to: 



M r 5% 

E 6 



F r 



M r q~ 
-eE 6 q~ 



(8) 



After the heel-strike, the foot rotates around the ankle joint. The EoM of the model is: 



Mr Ej Ej 



E 3 
E 6 





q 




' F z ' 




F cl 


= 


~"3 




fc2_ 




_ — "6. 



(9) 



Note that the constraint equations guarantee that the stance leg maintains foot contact 
with ground and the swing leg maintains heel contact with ground. In addition, the 
spring constant k in the compliant ankle joints should not be too big. Otherwise, the 
stance leg will lose contact with ground. In this phase, since the foot rotates around the 
ankle joint, the force generated by the springs on the swing leg should be considered as 
external force. Thus, in this event, the mass matrix M would not include the point mass 
of the swing foot. 



'M r Ef 

.-4 _ 




\q + ~ 


= 


M r q- 
-eSiq~ 
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2.4 Toe-Strike Impulse 

The proposed walking model with flat feet introduces a toe-strike impulse in addition 
to the heel-strike collision. The EoM of the model is: 

(10) 

Note that in this phase, we consider that the ankle joint of the swing leg is constrained to 
move in a circular orbit with toe as the center and distance between toe and ankle joint 
as the radius. The force generated by the spring on the swing leg should be considered 
as external force, which can also be considered as the constraint force of the circular 
orbit. After the toe-strike, one step ends. 

2.5 Effects of Hip Actuation and Ankle Compliance 

By application of the cell mapping method that has been used in the several studies of 
passive dynamic walking (e.g. |[T9l . 1211 '). we found that the model performs well in 
the concept of global stability. The allowable errors can be much larger than the results 
obtained in |fi9) . This can be inspected by the evaluation of the basin of attraction as 
shown in Fig.[2a), which is the complete set of initial conditions that eventually result 
in cyclic walking motion. One can find that cyclic walking with initial conditions in 
Table Q] emerges even if the initial step is nearly fourfold as large, e.g. {«i(0) = 0.8, 
di(0) = —2.3, d2(0) = —0.8}. It indicates that passive dynamic walking with flat feet 
and ankle compliance may play better on rough terrain with ground disturbance. 

It has been examined that optimized hip actuation can improve the stability of the 
passive walker [9|. In addition, one can use the hip actuation as the control parameter 
to achieve stable walking on uneven terrain [13|-[ 16|. In our model mentioned above, 
ankle compliance k can also be used as a control parameter to affect the walking gait. 
As shown in Fig. Htb), under the same change of hip actuation, different ankle com- 
pliance reveals different responses in walking velocity transition. It indicates that more 
compliance results in more visible sensitivity to the change of hip torque. According to 
the analysis of basins of attraction with different k [ 1 8 ] , we find that a relatively small k 
will lead to more stable points. However, more compliance in ankle joints may result in 
often falling backward during walking. Thus, optimized ankle compliance may result 
in a more stable bipedal walking that allows larger disturbance. 



3 Optimization Based Walking Control 

In order to optimize the hip actuation and ankle compliance which affect walking gait as 
analyzed above, Particle Swarm Optimization (PSO) has been chosen with a focus lying 
on quickly finding suitable results, in view of time-consuming and adaptivity of the gait. 
In the realization of the PSO algorithm, a swarm of N particles is constructed inside 
a D-dimensional real valued solution space, where each position can be a potential 
solution for the optimization problem. The position of each particle is denoted as Xi 
(0 < i < N). Each particle has a velocity parameter Vi (0 < i < N). It specifies 
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(a) 



- k=1 .65 Nm/rad 

- k=8 . 65 Nm/rad 
-k=15 Nm/rad 

- k=25 Nm/rad 

- k=35 Nm/rad 
stiff ankle 



(b) 



Fig. 2. (a) Basin of attraction of the passive dynamic walking model with flat feet and compliant 
ankles. The blue layers of points represent horizontal slices of a three-dimensional region of 
initial conditions that eventually result in the cyclic walking motion. The fixed point is indicated 
with a red point, which is above one of the sample slices, (b) Results of actively changing walking 
speed with the same hip actuation and different ankle compliance (k varies). 



that the length and the direction of Xi should be modified during iteration. A fitness 
value attached to each location represents how well the location suits the optimization 
problem. The fitness value can be calculated by a fitness function of the optimization. 

In this study, we used adaptive PSO with changing inertia weight. The update equa- 
tion for velocity with inertial weight is described as follows: 



v u X = wv *d + cir k ld (pbest k ld - x k d ) + c 2 r k d (gbest k - x k d ) 



(11) 



where w is the inertia weight. v k d is one component of Vi (d donates the component 
number) at iteration k. Similarly, x k d is one component of Xi at iteration k. pbesti 
(0 < % < N) and gbest are the personal best position and the global best position at each 
iteration respectively. c\ and c 2 are acceleration factors. r\ and r 2 are random numbers 
uniformly distributed between and 1 . Note that each component of the velocity has 
new random numbers. In order to prevent particles from flying outside the searching 
space, the amplitude of the velocity is constrained inside a spectrum [— v™ a:E , +v 7 ^ 



..maxi 
J d J 



3.1 Fitness Function and Optimization Process 

For a specific passive walker, the mechanical parameters (length and mass distribution) 
are constant. To control passive dynamic walking on uneven terrain, we focus the con- 
trol parameters on hip actuation P and ankle compliance k. Then the two-dimensional 
parameter space is (P, k). Here a set of parameters stands for a particle of PSO. Since 
the walker will be optimized with integration of stability and efficiency, the fitness func- 
tion is defined as follows: 

a = a s +"/a e (12) 

where a s and a e are the fitness value to assess the stability and efficiency of each set 
of parameters respectively. 7 is the tuning factor to change the importance of the two 
characteristics. 
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There are several methods to evaluate the stability of the passive dynamic walking. 
In this study, the stability will be quantified by the modulus of the Jacobian matrix of 
the mapping function as defined in IT9l . Here, we notate the maximal eigenvalue as 
A m , which represents the decreasing speed of the deviation. The stability grades varies 
for different sets of parameters even they all have a stable fixed point. The smaller the 
A m is, the faster the deviation decreases and the more stable the walker is. The similar 
conclusion can be obtained when all sets of parameters only have an unstable fixed 
point. The larger the X m is, the more far from the stable state the walker is. Then we 
define the fitness function of stability as the follows: 

a. = ^- (13) 

Am 

Similar to [2] and [19|, the energetic efficiency of walking can be evaluated by the 

specific resistance as follows: 

jp 

6 = (14) 

MgL 

where E is the cost of energy. In this study, the energy cost is generated only by the hip 
torque. M is the total mass of the model, g is the acceleration of gravity. L is the length 
of distance the robot passed. Then the fitness function can be defined as follows: 

1 M 9 l n^ 

ae = 0=— (15) 

From ( TT21 . ( TT3T ) and (TT3T >. we can obtain the whole expression of the fitness function: 

1 Mql 

a=— + 7 -f (16) 

Additionally, to evaluate the walking motion after overcoming ground disturbance, we 
introduce another expression of fitness function. We use further walking distance d 
instead of the fitness efficiency er e . Then the fitness function can be rewritten as: 

a = ^-+ 7 d (17) 

3.2 Gait Controller with Optimized Parameters 

After analyzing the effects of hip actuation and ankle compliance in the stability and 
adaptability of the passive dynamic walker, we select P and k as the gait control input. 
The output of the optimization simulator P (t) and k (t) are added to the current actual 
hip actuation P a (t) and ankle compliance k a (t) (see Fig. 01. This results in extra hip 
torque to move the swing leg more forward and prevent tripping. The purpose of the 
failure-detection block in Fig. [3] is to monitor in simulations the foot contact and the 
knee locking in order to detect whether walking failed. A failure means that the robot 
fell either backward or forward or that it started running (both feet leave contact with 
ground). There is a active control counter module in the diagram. It is used to count the 
times of applying active control (increasing or decreasing P or k) during one contin- 
uous walking. The output of this module make the simulator change P or k. With the 
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Fig. 3. The control scheme to overcome ground disturbance with optimized hip torque and ankle 
compliance 



dynamic model that adequately describing the real robot, an adaptive optimizing con- 
trol scheme can be done without manually set to teach the robot and without the robot 
damaging itself. 



4 Experimental Results 

All the simulation experiments used the dynamic model mentioned in Section II which 
was implemented in MATLAB, using the parameter values shown in Table Q] The nu- 
merical integration of the second order differential EoMs uses the Runge-Kutta method, 
which is similar to the simulation methods mentioned in 11181191 . 



4.1 Parameter Optimization 

Based on the adaptive PSO with proper inertial weight mentioned above, we optimized 
the hip torque (hip actuation) and ankle compliance to achieve adaptive walking with 
maximal allowable ground disturbance of the model with parameter values in Table Q] 
The testing scenario is a floor with one step down. The height of the step is gradually 
chosen from the range from 1mm to 20mm. The initial particles which represent the 
parameter set of P and k are randomly selected from the corresponding points in|2ja) 
that will finally achieve stable walking. During the walking simulation of the dynamic 
model, the ground disturbance gradually increase. The optimization process evaluates 
the maximum ground disturbance of the dynamic model with certain P and k. Dur- 
ing the walking, the selected P and k keep constant to overcome gradually increased 
ground disturbance. The optimization finally record the maximum ground disturbance 
in each generation. The fitness function is (fT6t . Fig.[T2]shows the results. It is clear that 
by applying the adaptive PSO, the optimization process can quickly find the optimal 
parameter set. It also indicated that with optimized hip actuation and ankle torque, the 
passive dynamic walker can achieve stable walking with no active control even if there 
is —11mm ground disturbance. 
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Fig. 4. Maximum ground disturbance with optimized hip torque and ankle compliance that keep 
unchanged during stable walking 



4.2 Walking on Uneven Terrain with Control 

Starting from the optimized P and k with no active control during walking simulations, 
we add the control scheme shown in Fig. [3] to the walking motion. A preset ground 
disturbance occurs at known time during the level walking. The active control counter 
determines the times of active control to change P and k. In this simulation, the ground 
disturbance varies from —15mm to —25mm. The fitness function of the optimization 
is dTTb . The counter first makes the optimization simulator to actively change P and k 
once when ground disturbance occurs. There was no optimized set of P and k that can 
overcome the —25mm step. Then the counter makes the simulator to actively change P 
and k twice when ground disturbance occurs. The walker successfully achieved stable 
walking with —25mm disturbance (see stick diagram shown in Fig.|3a)). Fig.[5jb) and 
(c) show the trajectories of hip and knee during the adaptive walking. Note that the 
cyclic walking was initially actuated by a relatively small hip torque. After one time 
of active control (varying P and k), the hip torque increased to move the swing leg 
more forward and prevent tripping. Since there was a second time of active control, the 
trajectories of the swing thigh and the swing shank transited to bigger limit cycles. Such 
optimized P and k finally stabilized the walking motion after a step down occurred. 

Fig.|6]shows the optimization process of the hip actuation and ankle compliance dur- 
ing the walking with two times of active control. We set that if the walker can walk sta- 
bly for enough time after the ground disturbance, the walking motion is adaptive on the 
uneven ground. Specifically, the distance is the product of walking speed times 25 sec- 
onds. Fig.[6ja) demonstrates the results of further distance after ground disturbance each 
generation. After four generations, the walker can find optimized parameters to over- 
come the step. Fig.|6tb) shows the process of selecting P and k during the optimization. 
The initial hip torque is 5.5130iVm and ankle compliance is 12.2218Nm/rad. The 
two times of variance of P and k are (18.0000iVm, 16.0000Nm/rad and (5.5000iVm, 
15A228Nm/rad) respectively. Though there is no complex learning algorithm in the 
control scheme, the walker can perform better terrain adaptability comparing with other 
simulation results (e.g. fl"3l . (TTl ). 
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igle of thigh (tad) 



(a) 



(b) 




(c) 

Fig. 5. Adaptive locomotion with active control on uneven terrain. This result is obtained every 
10 frames during a continuous walking, (a) is the stick diagram, (b) and (c) are the angular 
trajectories of the thigh and shank respectively. 





ankle compliance (Nm/raC) 



(a) 



(b) 



Fig. 6. Optimization of hip actuation and ankle compliance. Both P and k vary during the contin- 
uous walking, (a) is the results of further distance after ground disturbance each generation, (b) 
is the results of selecting hip actuation and ankle compliance during optimization. 



4.3 Real Robot Experiments 

To study natural and energy-efficient bipedal walking, we designed and constructed a 
bipedal robot prototype , 1.2m in height and 20fcg in weight. With the bisecting hip 
mechanism similar to l2ll . the prototype has five Degrees of Freedom (DOFs). Two 
commercial motors are used in the hip joints to perform hip actuation. Each leg consists 
of a thigh and a shank interconnected through a passive knee joint that has a locking 
mechanism. Springs are installed between the foot and the plate that is pushing the leg 
up while it is rotated around the ankle. To prevent foot-scuffing, we modified the foot 
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Fig. 7. A sequence of photos captured during autonomous walking of the robot prototype on 
natural ground 

with arc in the front and the back-end. Specific mechanical parameters are shown in 
Table Q] In the experiment, by using the proposed method, the robot tried to walk on 
natural ground outdoor. The natural ground is not a strict continuous level floor, where 
random irregularity of the ground and slight slippery occurred. 

Fig. [7] shows the result. The robot can achieve three-dimensional stable walking on 
natural ground with more than 10 steps. Comparing to the results of terrain adaptability 
of other real robot experiments (e.g. two-dimensional walkers [20], |16|), the successful 
three-dimensional walking of the robot prototype shows that the quasi-passive dynamic 
walker with optimized hip actuation and ankle compliance can perform stable walking 
with larger ground disturbance. 

5 Conclusions 

In this paper, we have investigated how to control passivity-based bipedal walkers to 
achieve stable locomotion with terrain adaptability. Satisfactory results of simulations 
and real robot experiments indicated that having the fixed mechanical parameters dur- 
ing walking, the passivity-based walker can walk on uneven terrain with larger ground 
disturbance by optimized hip actuation and ankle compliance in view of walking sta- 
bility and efficiency. In the future, more real robot experiments will be continued to 
overcome more complex ground disturbance. 
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Abstract. This paper deals with the decentralized navigation problem for multi- 
robots systems. The proposed planning scheme consists of two parallel processes. 
A decentralized receding horizon motion planner, in which each robot computes 
its planned trajectory, enables the coordination between cooperative robots. A fast 
navigation controller, based on artificial potential fields and sliding mode control 
technique, is used to simultaneously track the planned trajectory while avoiding 
collision with unexpected entities. Simulation studies are provided in order to 
show the effectiveness of the proposed approach. 

Keywords: Collision avoidance, Decentralized path planning, Multi-robots sys- 
tem, Potential field, Sliding mode control. 



1 Introduction 

The research effort in multi-robots systems (MRS) relies on the fact that multiple robots 
have the possibility to perform a mission more efficiently than a single robot. Among all 
the topics of study in this field, the issue of conflict resolution becomes an increasingly 
important point. Many cooperative tasks such as surveillance, search, rescue or area 
data acquisition need the robots to autonomously navigate without collision. 

Solving conflicts in MRS consists in introducing some coordination mechanisms 
in order to give a coherence between the robot acts (Tj. For motion planning, three 
coordination mechanisms are identified: 

- the coordination by adjustment, where each robot adapts its behavior to achieve a 
common objective [2|. However, most of the planning algorithms are centralized, 
which often limit their applicability in real systems. 

- the coordination by leadership (or supervision) where a hierarchical relationship 
exists between robots 13. Such an approach is easy to implement [4|. However, 
due to the lack of an explicit feedback from the followers to the leader, the col- 
lision avoidance cannot be guaranteed if followers are perturbed (during obstacle 
avoidance for instance). Another disadvantage is that the leader is a single point of 
failure. 
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- the standardization where procedures are predefined to solve some particular in- 
teraction cases [5j. While this approach can lead to straightforward proofs, it also 
tends to be less flexible with respect to changing conditions. 

Here, the problem of interest is the decentralized navigation for autonomous robots 
through a coordination by adjustment. Each vehicle is modeled as an unicycle with 
a limited sensing range in order to capture the essential properties of a wide range of 
vehicles. They are dynamically decoupled but have common constraints that make some 
conflicts. Indeed, each robot has to avoid collision with the other entities. Furthermore, 
the proposed framework allows moving (and static) obstacles to be avoided since they 
can be modeled as non cooperative entities. 

Motion planning consists in generating a collision-free trajectory from the initial to 
the final desired positions for a robot. Since the environment is partially known and 
further explored in real time, the computation of complete trajectories from start until 
finish must be avoided. Therefore, the trajectories have to be computed gradually over 
time while the mission unfolds. It can be accomplished using an online receding hori- 
zon planner ||6), in which partial trajectories from an initial state toward the goal are 
computed by solving optimal control problems over a limited horizon 1 7 1 . 

Two strategies for motion planning in MRS are the centralized and decentralized 
(distributed) approaches. Although the centralized one has been used in different studies 
(see [8] for instance), its computation time which scales exponentially with the number 
of robots, its communication requirement and its lack of security make it prohibitive. To 
overcome these limitations, one can use a distributed strategy which results in behaviors 
closed to what is obtained with a centralized approach. 

Recently, some decentralized algorithms are based on a receding horizon approach. 
In J9|, a distributed solution is provided for the rigid formation stabilization problem. 
In JT0|, the decentralized receding horizon planner is solved using mixed-integer lin- 
ear programming (MILP). Every robot is allotted a time slot to compute its own dy- 
namically feasible collision-free trajectory. This scheme creates a queueing order of 
non-conflicting groups of vehicles, where each group updates their trajectory sequen- 
tially and communicates the solution to its neighbors. In 1 1 1 1, a decentralized algorithm 
based on a coordination by adjustment is proposed to solve the navigation problem for 
MRS. However, the large amount of information exchanged between robots and the ad- 
dition of several constraints make this strategy prohibitive when the number of vehicles 
increases. 

One of other collision avoidance algorithm is potential field method, where an arti- 
ficial potential function treats each robot as a charged particle that repels all the other 
entities lfT2ll . However, most of them are only based on relative position information 
and do not consider coordination between cooperative robots. 

In this paper, we proposed a practical decentralized scheme, based on a coordination 
by adjustment, for real time navigation of large-scale MRS. As illustrated in Fig.Q] the 
scheme consists of two parallel processes: 

- a distributed receding horizon planner, in which each robot computes its own 
planned trajectory locally, for the coordination between cooperative robots, 
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- a reactive approach, which combines artificial potential fields and sliding mode 
control technique, for simultaneously tracking the planned trajectory while avoid- 
ing collision with unexpected entities (i.e. non cooperative entities). 

The main advantages of the proposed strategy, especially for large-scale MRS, are the 
small amount of information exchanged between cooperative robots and the robustness. 



actual position and velocity 



of cooperative robots 



Receding horizon f \ 

{ Process 1 

planner v__ 



actual position and velocity 

of non cooperative entities / 
(obstacles, moving objects) 



planned trajectory 
planned velocity 



Reactive navigation f ^ 
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controller \^_ 



Robot 5\n 



real control inputs 



Fig. 1. Proposed navigation algorithm 

The outline of this paper is as follows. In Section 2, the problem setup is described. 
In Section 3, the navigation algorithm is presented. Finally, numerical results illustrate 
the effectiveness of the strategy. 

2 Problem Statement 

2.1 Dynamic Model of the Robots 

Each robot A n (n € (1, . . . , N) with N e N), shown in Fig.[2j is of unicycle-type. Its 
two fixed driving wheels of radius r n , separated by 2p n , are independently controlled 
by two actuators (DC motors) and the passive wheel prevents the robot from tipping 
over as it moves on a plane. The centre of mass C n , whose coordinates are (x n , y n ), is 
located at the intersection of a straight line passing through the middle of the robot and 
the axis of the two driving wheels. Its configuration is given by: 



Vn 



[*^n ) Vnj "n\ 



where (x n ,y n ) is the position of its mass center C„ and 9 n is its orientation in the 
global frame. The kinematics of the robot is shown under the nonholonomic constraints 
lfT3l . The pure rolling and nonslipping nonholonomic conditions are described by: 



[-smd n ,cos9 n ,0]r) n = 
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Fig. 2. Unicycle-type robot 



The dynamic model of robot A n is given as in lfT4ll : 

Vn = J{Vn)z r , 

IVl n Z n H- U n Z n T n 

where 

- M n is a symmetric positive definite inertia matrix 

- D n is a symmetric damping matrix 

- the transformation matrix J(rj n ) is 



•HVn) = -J 



coao„ cost 



sin t) n 

Pn 1 



sine 



-Pn 



(1) 

(2) 



(3) 



JlT 



where z r n , z n are the angular velocities of the right and left wheels. 
The relationship between z n and the linear and angular velocities, denoted v n ,w n , 
is 



J 



B,- 



with B„ 



1 



1 Pn 
1 -Pn 



(4) 



- T„ = 

robot 



[ T n' T n\ T where t^t^ are the control torques applied to the wheels of the 



Remark 1. System dZJ-([2]) is flat (see [15] for details about flatness) since all system 
variables can be differentially parameterized by x n , y n as well as a finite number of 
their time derivatives. For instance, n , v n and w n can be expressed as 



fin 



arctan^, v n = \/x% + y% 

%n 



II!,: 



Vn^n ^nVn 



•■*■■ i: 



!Jn 



Remark 2. Speed u n = [i„, y n ] of A n is restricted to lie in a closed interval S n 



S„ = { U n £ M 2 | ||tt„|| < U n .rnax } 



(5) 
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2.2 Assumptions and Control Objective 

Assumption 1. The following assumptions are made: 

- A n knows its position p n = [x n , y n ] T and its velocity u n = [x n , y n ] T 

- A n has a physical safety area, which is centered at C n with a radius R n , and has a 
circular communication area which is also centered at C n with a radius R n - Note 
that R n is strictly larger than R n + Rj, j G (1, . . . , N),j =/= n 

- A n broadcasts (p n ,u n ) and receives (pj,Uj) broadcasted by other cooperative 
robots Aj, in its communication area 

- An can compute the relative position and velocity (Pobst , Uobs t ) ofnon cooperative 
entities within a given sensing range 

- At the initial time ti n i > 0, each robot starts at a location outside of the safety 
areas of other entities. 

The objective is to find the control inputs x„ for each robot A n such that, under As- 
sumptionQ"! 

- A n is stabilized toward its desired point p n .des, i.e. 

lim \\p n (t) - Pn.des || = (6) 

t — >oo 

- collisions are avoided 

- all computations are done on board in a decentralized cooperative way 

Remark 3. It should be noted that for collision avoidance, one can distinguish two 
kinds of entities, i.e. 

- cooperative robots which are involved in a detected potential collision 

- non cooperative entities which cannot cooperate in the collision avoidance process. 
They represent the moving objects and static obstacles. 

3 Distributed Algorithm 

In order to solve the multi-robots navigation problem, a decentralized algorithm com- 
bining two parallel processes is proposed. First, a receding horizon planner, in which 
each robot computes its own planned trajectory locally, achieves middle-term objec- 
tives, i.e. coordination between cooperative robots which are involved in a detected po- 
tential collision. Then, a reactive navigation controller is proposed to fulfill short-term 
objectives, i.e. trajectory tracking while taking into account non cooperative entities. 

3.1 Conflicts and Collisions 

Definition 1. (conflict) A conflict occurs between two cooperative robots A n and Aj 
at time tk £ M + z/ they are not in collision at tk, but at some future time, a collision 
may occur. 

The following proposition, based on the well-known concept of velocity obstacle lfl6ll . 
is useful to check the presence of conflicts. 
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Fig. 3. Velocity obstacle concept 



Proposition 1. Let us define for each pair (Am Aj), the following variables depicted 
in Fig. ]3\ 



Pnj{tk) = arg(M„(i fe ) - Uj(tk)) - arg(pj(i fc ) -p„(i fe )) 

U \ ■ ( R n +Rj 

ttnj{tk) = 



arcsm 



(7) 



,l|Pi(tfc)-Pn(t*)ll 

A necessary and sufficient condition for no conflict between A n and Aj at tk is: 

\Pnj(tk)\ > OnjCtfc) (8) 



Definition 2. (conflict subset) For each A n , the conflict subset J\f n (tk) at time tk £ M + 
is the set of all cooperative robots which are in the communication area of A n and in 
conflict with A n - 



3.2 Receding Horizon Planner 

In this section, we introduce a decentralized receding horizon planning scheme used 
to achieve middle-term objectives, based on 117I11L The purpose of the distributed 
receding horizon planner is to decompose the overall problem into a family of simple 
receding horizon planning problems which are implemented in each robot A n . In every 
problem, the same planning horizon T p e R + and update period T c e K + (T c < T p ) 
are used. The receding horizon updates are: 



tk = U m + (fc - 1)T C) k e N 



(9) 



Remark 4. During the initialization step, that is to say before robots move, we denote 

At each update tk, robots in conflict exchange information about each others (position, 
velocity, etc.) to predict their possible trajectories, denoted by Pj(t, tk) and velocities 
Uj(t,tk) (for all Aj belonging to the conflict subset). It is similar as what it is done 
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while driving a car. These trajectories are obtained without taking the collision avoid- 
ance constraint into account. Therefore, by design, the anticipated trajectory is the same 
in every receding horizon planning problem in which it occurs. At last, in parallel, every 
robot A n computes its own planned trajectory p n (t,t k ) and planned velocity u n (t,t k ), 
over the planning horizon T p , in order to integrate the collision avoidance between co- 
operative robots. From the planned trajectory and velocity associated to the planning 
horizon T p , only the part which corresponds to the update horizon T c is stored. At time 
tk+i, a new optimization is solved over a shifted planning horizon. 

Remark 5. Note that the first argument ofp n , p n , u n and u n denotes time. The second 
argument is only added to distinguish at which receding horizon update the trajectory 
and velocity are computed. 

The collection of distributed receding horizon planning problems is formally defined, 
for each robot A n , as follows: 

Problem: At each update tk, for each robot A n : 

Given : the actual positions p n (tk), Pj(tk) an d the actual velocities U n (tk), Uj(tk) of 

robot A n and robots Aj belonging to Af n (tk), respectively. 

Find : the planned trajectory and velocity pairs (p*(t,t fc ),u*(t, t fc )), vt e [t k +i,t k+1 +t p ], 

that minimizes: 

rt k+1 +T p / _ \ 

/ [in J! Un 3 ,rep(t) + \\p n (t,t k )-pn(t,t k )\\\dt (10) 

tk+1 \ iGN 1 4je/V n (tj0 / 

subject to the following constraints: Mi £ {i £ N | Ai £ N n (t k ) U {-4n}}, 



Pi(tk,tk) =Pi(tk) Ui(t k ,t k ) = Ui(t k ) 
Pi\ tfinj T>k) — Pi,des l^iyt finj^k ) — ^i^des 

Ui(t,tk) £ Si , Vt > tk 



(11) 



and 



where 
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(12) 



(13) 



a„ and b n are strictly positive factors which can vary among robots to reflect differences 
in aggressiveness (a n < 1, b n <C 1) and shyness (a„ > 1, b n 3> 1)- 

It could be highlighted that the anticipated trajectories are computed without taking 
the collision avoidance constraint into account (see Eq. (fTTll). That is why, to inte- 
grate the path planning with local collision avoidance, the cost function i fTOb should be 
appropriately chosen. 
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Anticipated trajectory 
Planned trajectory 




Fig. 4. Implementation of the receding horizon planner 



One can note that the first part of cost ( fTOb is designed to enforce the collision avoid- 
ance between cooperative robots. This term, based on artificial potential field lfT2l . is 
designed such that it equals to infinity when a collision between robots occurs and de- 
creases according to the relative distance between robots. 

The second part of cost (fTOb. i.e. the term ||p* (t,tk) — p n (t,tk)\\, is away of penal- 
izing the deviation of the optimal planned trajectory p* (t, tk) from the anticipated tra- 
jectory p n (t, tk), which is the trajectory that other robots rely on. In previous work, this 
term was incorporated into the decentralized receding horizon planner as a constraint 
fm . The formulation presented here is an improvement over this past formulation, 
since the penalty yields an optimization problem that is much easier to solve. 

Remark 6. One can note that constraints ( 1721 ) which guarantee the continuity of the 
planned trajectory and velocity need p* (tfe+i, tk— l) and lljj (tk+l, ife-i) computed in 
the previous step. Therefore, the proposed planner is not able to reject external distur- 
bances or inherent discrepancies between the model and the real process. However, it 
takes the real time constraint into account. Indeed, each robot has a limited time to plan 
its trajectory. The time allocated to make its decision depends on its perception sensors, 
its computation delays and is less than the update period T c (see Fig. [4J- 

The discussed claim for robustness in trajectory tracking will be achieved hereafter. 

Remark 7. A compromise must be done between reactivity and computation time. In- 
deed, the planning horizon must be sufficiently small in order to have good enough 
results in terms of computation time. However, it must be higher than the update period 
to guarantee enough reactivity. 
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Remark 8. To numerically solve the above problems, a nonlinear trajectory generation 
algorithm / 7/ is applied. It is based on finding trajectory curves in a lower dimensional 
space and parameterizing these curves by B-splines. A constrained feasible sequential 
quadratic optimization algorithm is used to find the B-splines coefficients that optimize 
the performance objective while respecting the constraints. 

3.3 Reactive Navigation Controller 

Hereafter, a reactive approach, which combines artificial potential fields and sliding 
mode control technique, for simultaneously tracking the planned trajectory while avoid- 
ing collision with unexpected entities (i.e. non cooperative entities), is proposed. 

Since the robot dynamics (Q]l-(|2]l is of strict feedback systems (see [18] for details 
about strict feedback systems) with respect to the robot linear and angular velocities 
(i.e. v n and w n ), a backstepping procedure is used to design the control input r n . That 
is why the control design is divided into two main steps. 

Step 1 based on Artificial Potential Fields. Let us introduce the following notations: 

One = #n - J6 n q^ 

where 7g ra and j Vn are auxiliary variables used to avoid collisions. Replacing expres- 
sions (TBI into the first two equations of (fl} and using Q yield: 



Pr, 



cos 7e n 
sm 70„ 



lvn + A ln + A 2n (15) 



with A ln = j Vri 



(cos6 ne - 1) cos7e„ - sin6»„ e sin 7^ 
sin6> ne cos 7^ + (cos6> ne - 1) sin 7^ 



and Ay 



cos9„ 
sm9 r , 



The objective is to design the auxiliary variables 7„ ra and 7e„ such that robot A n ro- 
bustly tracks its planned trajectory p* while avoiding unexpected collisions. Here, ar- 
tificial potential functions are used in order to design an attractive force between the 
robot and its planned trajectory and a repulsive force to avoid collisions. 

In conventional potential field method [12], the planned robot velocity m* is assumed 
to be zero and the obstacle velocity u b Si is not considered. However, to make robot A n 
track the planned trajectory among moving obstacles, velocities w* and u b Si play key 
roles. This issue will be addressed by extending the results given in [19]. Let us consider 
the conventional potential function lfT2ll : 

U n = U n>a tt + U ntre p (16) 

where U n , a tt and U n , rep are, respectively, the attractive potential defined to track the 
planned trajectory p* and the repulsive potential related to collision avoidance, speci- 
fied as follows: 

- The attractive potential is designed such that it puts penalty on the tracking error 
and is equal to zero when the robot is at its desired position, i.e. 

Un,att = \\\Pn-p* n \\ 2 (17) 
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- The repulsive potential is designed such that it equals to infinity when a collision 
occurs with A n and decreases according to the relative distance between A n and 
an obstacle, i.e. _ 

i 

with 

[0 if p ni > d n 

U nii rep = \ x ( _i L \ 2 ^^ (19) 



{ 2 yp^i d n 

where p r „; is the minimum distance between robot A n and the obstacle i. c n and d n 
are strictly positive factors which have similar properties as a n and b n . 

Proposition 2. If the errors 9 ne and v ne are asymptotically stable, A n robustly tracks 
its planned trajectory p n while avoiding collisions using the auxiliary variables: 

~/v„ = [(||u*||coB(e; -*„)- c„Ei«„ill«o6 3i l|co S (8 oi;s . -*„j)+ ||p„ -p* ||) 2 + |K*|| 2 Bin 2 (e* -^„)] " 5 

| i, n + arcsin ( U<\\°W-i>-) ) if y -L 

l e* n if 7,„ = o 

(20) 

with 

9* n = arg«) 
Oobsi = arg(« o6s J 
V>„ = arg (p* n - p n ) 
ip ni = arg (p obSs - p n ) 

j if p m > d n 

*™ = I f— " t- ) , \*n - »ii <*« 
Proof. Let us differentiate [7„ with respect to time in equation (fT6l l. i.e.: 

fn = U n>a tt + U n ^ re p (21) 

Substituting ( f20b into (I2TI 1 yields after some geometric manipulations: 

U n = ||K - PnlKIKH cos(6>* - V») - J Vn cos( 7eri - V„)- 

Cn Si Cm (||«o6si || COs(6> obs , - 1p ni ) + J Vft COs(jg n - Ipni))) 

+ ((Pn ~ P„F " Cn Ei ^ |lS-pl!l| ( P " ~ P°" S J T ) (An + An) 

= llPn - Pn II (IK II cos(0*-^ n ) - 7„„ cos( 7e7i -V„) 

_C n Si Cm (||Wofe Sj || COs{9 obsi - Ipni))) 

+ ((Pn - P„F " C„ £, ^ |£-p!:.''|| (P" - Po^J T ) (An + An) 

= Up; - PnlKIKII cos(0; - ^) - ^MW^i)- 

Cn Si Cm (||u 6 Si || COs(6> o6s! -Ipni))) 

+ (fPn ~ PnV ~ Cn E, Cm [feg | | (P» ~ Pob Sl ) T ) (An + An) 
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Assuming that the errors 6 ne and v ne are asymptotically stable (i.e. A\ n = A<z n = 0), 
one can get from (l20l l: 

Un^-Wpl-PnW 2 (22) 

Since U n > and U n < 0, U n is bounded. That is why A n robustly tracks its planned 
trajectory p* while avoiding collisions. D 

Step 2 based on Sliding Mode Technique. Now, the objective is to force the motion 
of robot A n such that the errors 6 ne and v ne are asymptotically stable. The proposed 
strategy is based on the so-called second order sliding mode control (SMC) approach. 
The SMC methodology [20| is chosen because it is a robust technique to control non- 
linear systems operating under uncertainty conditions [21]. Furthermore, second order 
SMC can reduce the chattering phenomenon (high frequency vibrations of the con- 
trolled system which degrade the performances). Indeed, instead of influencing the first 
sliding variable time derivative, the signum function acts on its second time derivative. 
This method can also achieve a better convergence accuracy with respect to discrete 
sampling time than conventional SMC (see IT231 for a survey). 

Let us apply to system (Q]l-© the following preliminary feedback: 

f„ = {MnBnY 1 (r n - D n z n ) (23) 

where f„ = [fi n ,f2„] T is the auxiliary control input. Thus, system (Q])-© can be 
expressed as follows: 



Vn = J(r] n )B n 



(24) 
(25) 



Since the relative degree of system d24t-d25l> with respect to the sliding variable v ne is 
only one, a dynamic extension is done before designing the control (see [22] for further 
details). Thus, an integrator chain is added on the input variable fi„. 

There are several algorithms to ensure the finite time stabilization of the sliding vari- 
ables 8 ne and v ne towards the origin. Among them, the sampled twisting algorithm 11231 
has been developed for systems with relative degree two. This algorithm provides good 
convergence accuracy and robustness properties. It does not require the knowledge of 
the time derivative of the sliding variables and takes into account some practical con- 
straints such as the sampling of the measurement and the control. 

Proposition 3. Consider system (Q-([2]). The errors ne and v ne are stable infinite time 
under the nonlinear controller defined in Ii23\l where 

-Ai jM sign (6> ne ) if ne Ae n(i > 
-Ai jTO sign(6' ne ) if ne Ag nc < 
-A 2 ,m sign (v ne ) if V ne A Vne > 
-A 2 , m sign(w„ e ) if v ne A Vne < 
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with 

1)T S ) else 



, ,, ifk = 



A 




ifk = (27) 



1)T S ) else 

T s is the sampling period, k G N is related to the time of the process and Xi_ rn , Xi^m, 
i = 1, 2 are positive constants high enough to enforce the sliding motion. 

Proof. It can be shown that this controller ensures a finite time convergence of the 
trajectories onto the manifold {v ne = v ne = 0} and < 9 ne = 9 ne = >. Hence, the ap- 
plication of the control input d26] l results in the robust finite time stabilization of 9 ne 

and v ne . □ 

Remark 9. We would like to emphasize that although not explicitly considered here 
the procedure based on sliding mode control guarantees proper behavior even in the 
presence of uncertainties in the mass and inertia of the robots and additive disturbances 
to the linear and angular velocities which constitute very realistic assumptions. 

Once the sliding mode occurs on all the surfaces (which happens in finite time), based 
on Proposition 2, the global control objectives, defined in Section 2.2, are fulfilled. 

Some specific advantages of the proposed decentralized algorithm are enumerated 
below: 

- robustness with respect to uncertainties and disturbances (sliding mode controller), 

- reactivity (potential field functions), 

- low communication bandwidth, i.e. small amount of information is locally ex- 
changed, 

- reduction of deadlocks due to local minima in potential field (anticipation and co- 
ordination mechanism through the receding horizon planner). 

4 Simulation Results 

This section demonstrates the performance of the proposed decentralized algorithm. 
The following simulations showcase two different scenarios for which the environment 
is partially known (i.e. the range of sensors of each robot is of radius 1.5m). The main 
parameters of the robots are: Vn, R n — 0.25m, R n — Am and «„ jmnl = lm/s. For 
the decentralized algorithm, the following parameters are used: T p = 3s, T c — 0.5s, 
a„ = c n = 1, b n = 2, d„ = 0.5, Ai,m = A 2 ,m = 10, Ai, m = A 2 . m = 1 and 
T s = 0.01s. 

4.1 Scenario 1: Crossing 

In this scenario, there are four robots (N — 4) starting at p\(ti n i) = [5,0} T ,p2(ti n i) = 
[15, 0] T , P3(ti n i) = [10, 5] T and p4(ti„i) = [10, —5] T respectively, with velocities 
equal to zero. These robots must cross each other in order to reach their desired config- 
uration piA es = [15, 0] T , p 2 ,des = [5, 0] T , p 3 ,des = [10, -5] T and p^ des = [10,5] T . 
One can note that this problem is not trivial due to its symmetry properties. 
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Fig. 5. Four vehicles simulation: (a) Robot trajectories, (b) Relative distances between Ai and 
other robots. 

The simulation results are given in Fig. [5] One can see that each robot modifies its 
trajectory in order to avoid collision. Figure PJJb) depicts the evolution of the distance 
between robots. Since it is higher than 0.5m, the collision avoidance is guaranteed. 



4.2 Scenario 2: Reconfiguration with Collision Avoidance 

In this scenario, a swarm of five robots (N = 5) reconfigures its geometric shape 
(from "linear" to "triangular") while avoiding collisions with obstacles. The proposed 
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Fig. 6. Collision avoidance of five robots 
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decentralized controller has only a limited knowledge of the obstacles (initially un- 
known). It simply keeps the robots spaced out using the proposed potential field tech- 
nique. The five robots make decisions in order to avoid collisions. One can note that the 
number of potential conflicts is high. 

One can see in Fig. [6] that under the proposed decentralized algorithm, the robots 
meet the objective defined in Section 2.2. Note that the radius of obstacles is increased 
by 0.25m (dotted lines around obstacles) to take the size of robots into account. 

5 Conclusions 

A new distributed strategy for the navigation of multiple autonomous robots is pre- 
sented. The proposed scheme combines a decentralized receding horizon motion plan- 
ner to satisfy middle-term objectives (coordination between cooperative robots) with a 
fast navigation controller based on artificial potential fields and sliding mode control 
technique to satisfy short-term objectives (collision avoidance and trajectory tracking). 
The fact that there is no leader increases the security and the robustness of the mis- 
sions. Simulation studies are provided in order to show the effectiveness of the proposed 
approach. 

In the future, it is planned to design real time observers to estimate the relative 
velocities between robots. 
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Abstract. The performance of large groups of robots is often limited by a com- 
monly shared resource. This effect, termed interference, can have a large impact 
on robotic swarms. This article studies the issue of interference in a swarm of 
robots working on a harvesting task. The environment of the robots is spatially 
constrained, i.e., there is a commonly shared resource, the nest, which limits the 
group's performance when used without any arbitration mechanism. The arti- 
cle studies the use of task partitioning for reducing concurrent accesses to the 
resource, and therefore limiting the impact of interference on the group's per- 
formance. In our study, we spatially partition the environment into two subparts, 
thereby partitioning the corresponding harvesting task as well. We employ a sim- 
ple method to allocate individuals to the partitions. The approach is empirically 
studied both in an environment with a narrow nest area and an environment with- 
out this constraint. The results of the task partitioning strategy are analyzed and 
compared to the case in which task partitioning is not employed. 



1 Introduction 

In collective robotics, interference is a critical problem limiting the growth of a group: 
the time each robot spends in non-task-relevant behaviors such as obstacle avoidance 
increases when the density of individuals rises — see e.g., Q]]. The performance on tasks 
that suffer from physical interference can typically be improved by spatial partitioning; 
for example, by keeping each robot in its own "working area". A known approach that 
uses this rationale is the so called bucket-brigade [2 3]. In this approach, robots hand 
over objects to robots working in the following area, until the objects reach their desti- 
nation. As tasks usually cannot be partitioned arbitrarily, this approach effectively limits 
the number of robots that can be employed in the task. A possible solution to this prob- 
lem, treating working areas as non-exclusive, raises other problems: How should indi- 
viduals be allocated to tasks? How can such an allocation help in limiting the amount 
of interference? 

In this paper, we study how task partitioning can help in reducing sources of interfer- 
ence. Additionally, we show a simple way to achieve self-organized allocation to such 
a task partition when using a robotic swarm. 

We use the foraging problem, one of the canonical testbeds for collective robotics, 
as a base for our studies. In our experiments, a swarm of homogeneous robots has to 
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harvest prey objects from a source area and transport them to a home area. In this study, 
we limit ourselves to a harvesting task that is pre-partitioned by the designer into two 
subtasks with a sequential interdependency. We study a simple threshold-based model 
of self-organized allocation and focus on two questions: Under which environmental 
conditions is it advantageous to partition the task? Can this partition reduce interfer- 
ence between group members? These questions are studied in two experiments using a 
simulated robot swarm. 

The paper is organized as follows. We first review related works in Sect. [2] In Sect. [3] 
we explain the task partitioning and the allocation method employed in this study. Sec- 
tion |4] gives the methods used in the experiments by describing the environments, the 
simulated robots, and the controller. In Sect. [5] the results of the experiments are given 
and discussed. Section[6]draws some conclusions and discusses future work. 



2 Related Work 

Interference has long been acknowledged as being one of the key issues in multi-robot 
cooperation |4). Lerman and Galstyan devised a mathematical model that allows a quan- 
tification of the interference and its effect on group performance [ 1 ]. Probably, the most 
thorough study was published by Goldberg, who identified several types of multi-robot 
interactions Q. Goldberg notes that one of the most common types of interference is 
physical interference in a central area, for example the nest. This kind of interference 
results from resource conflicts, in this case physical space, and can be arbitrated by 
either making sure that robots stay in different areas all the time or by employing a 
scheduling mechanism to ensure that robots use the same space only at different times. 

A simple method for reducing interference by using the first arbitration method men- 
tioned is the so-called bucket-brigade: robots are forced to stay in exclusive working 
areas and to pass objects to the following robot as soon as they cross the boundaries of 
their area [23]. Recently, this has been extended to work with adaptive working areas 
by Lein and Vaughan 0. To the best of our knowledge, current works concerned with 
bucket brigading only studied the influence of interference due to obstacle avoidance. 
Other sources of interference (e.g., object manipulation) were never studied, although 
they might have a critical impact on the performance of any task partitioning approach. 
To quote Shell and Mataric |3|: "If the cost of picking up or dropping pucks is signifi- 
cant [...], then bucket brigading may not be suitable." 

Task allocation for multi-robot systems is a wide field, which can be divided in inten- 
tional and self-organized task allocation. Intentional task allocation relies on negotiation 
and explicit communication to create global allocations, whereas in self-organized task 
allocation global allocations result from local, stochastic decisions. A formal analysis 
and taxonomy that covers intentional task allocation has been proposed by Gerkey and 
Mataric [7|. Kalra and Martinoli recently compared the two best-known approaches of 
intentional and self-organized task allocation [ 8 1 . 

The field of self-organized task allocation is in its early stages, as most studies tackle 
simple problems without task interdependencies. An exception is the work of Schei- 
dler et al, in which helper components of a computing system are allocated to tasks 
with sequential interdependencies by using a threshold based method [9|. Studies in 
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self-organized task allocation are mostly based on threshold-based approaches, taking 
inspiration from division of labor in social insects. Krieger and Billeter were among 
the first to propose threshold-based approaches in multi-robot task allocation H 1 01 . Isa- 
bella et al. used threshold-based task allocation in a multi-foraging task [11]. Similarly, 
Campo and Dorigo used a notion of the group's internal energy to allocate individu- 
als to a multi-foraging task fl2l . Finally, Liu et al. studied a multi-foraging task while 
focusing on the influence of the use of different social cues on the overall group perfor- 
mance 1131. 



3 Task Partitioning and Allocation 

In this work, we study a collective foraging task. By spatially partitioning the environ- 
ment, the global foraging task is automatically partitioned into two subtasks: 1) har- 
vesting prey objects from a harvesting area (source) and 2) transporting them to a home 
area (nest). Robots working on the first subtask harvest prey objects from the source 
and pass them to the robots working on the second subtask, which store the objects in 
the nest. These subtasks have a sequential interdependency in the sense that they have 
to be performed one after the other in order to complete the global task once: delivering 
a prey object to the home area. Robots can decide to switch from one subtask to the 
other, thus creating a task allocation problem: individual robots have to be allocated to 
subtasks and different allocations yield different performance. As a prey object has to 
be passed directly from one robot to the other, a robot usually has to wait some time 
before passing a prey object to or receiving a prey object from a robot working on 
the other subtask. This waiting time can therefore give an indication of the allocation 
quality for the respective subtask: if the waiting time is very long, there might not be 
enough robots allocated to the other subtask. Thus, the robots can use this waiting time 
to decide whether to switch subtask or not. Ideally, the waiting time should be the same 
for the two subtasks in order for the system to reach a stable state and deliver optimal 
performance. 

Our robots exploit a simple threshold-based model to decide when to switch task: 
when the waiting time i w is higher than a threshold 0, a robot switches its subtask. 
The robot's waiting time is a function of the average time the robots working in the 
other subtask need to complete their task. The task-completion time of a robot depends 
on two factors: 1) round-trip-time (i.e., distance to travel) and 2) time lost due to in- 
terference. Thus, the robot's threshold is a function of the round-trip-time and the 
interference of the robots in the other subtask. Therefore, the optimal task switching 
threshold depends on the task (i.e., time to harvest a prey object) and the environment 
(i.e., distance between the source and the nest). As the parameters of the environment 
are not pre-programmed into the robots, determining the optimal threshold can be a 
complex problem. In this work, we limit ourselves to a simple method for setting this 
threshold: at the start of the experiment, each robot draws a random threshold that is 
used as its task switching threshold throughout the experiment. 

In the following, we study the properties of this simple self-organized task allocation 
strategy, compare this strategy to a strategy without task partitioning, and analyze how 
it can help to reduce interference. We refer to the two strategies as partitioned and 
non-partitioned, respectively. 
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4 Methods 

This section describes the environments in which the experiments are carried out, the 
simulated robots, and the robot's controller. Additionally, we describe how we run the 
experiments and we introduce some metrics that we use to evaluate the properties of the 
system. 

4.1 Environments 

We study task allocation in two different environments. In these two environments, 
the nest is marked by a light source that can be perceived by all robots, thus providing 
directional information. The environment is spatially partitioned in two parts: the source 
is located on the left and the nest is located on the right side of the arena. We refer to the 
two sides of the arena as harvest area and store area, respectively. The exchange zone 
is located between these two areas. Robots working on the left side, called harvesters, 
gather prey objects in the source and move them to the exchange zone, where they pass 
them to the robots working on the other side. These are referred to as storers: their role 
is to transport prey objects to the nest and store them there. The nest, the source, and 
the exchange zone can be detected through environmental cues (ground color). 

At time t = 0, the robots are randomly placed in the harvest area. The experiments 
run for i max = 18, 000 time steps (a simulated time of one hour, with a time step 
length of 200 ms). The experiments are run in two different arenas (see Fig. [TJ. The 
first arena (Fig. QJ) is 4.125 m long with a width of 1.6 m at the source and exchange 
zone, whereas the nest is 0.4 m wide. The exchange zone is located 3.125 m away from 
the source. This arena is characterized by the presence of an area, critical for the task, 
in which high interference between robots can be expected (the nest). Thus, this arena 
is referred to as the narrow-nest environment. 

The second arena (Fig. [lb) has a rectangular shape: it is 3.75 m long and 1.6 m wide. 
Here as well the exchange zone is located 3.125 m from the source. The arena shape 
does not suggest the presence of any zone where interference can be higher than in other 
places. This arena is referred to as the wide-nest environment. 

The area of both arenas is 6 m 2 , 5 m 2 for the harvest area and 1 m 2 for the store area. 
The overall area is the same in the two arenas, so that the same group size results in the 
same robot density. Thus, results are comparable across the two environments. 

4.2 Simulation 

The experiments are carried out in a custom simulation environment that models ge- 
ometries and functional properties of simple objects and robots. Our robots' model is 
purely kinematic. Prey objects are simulated as an attribute a robot can posses and 
not as physical entities. Although the experiments are conducted in simulation only, 
the simulated robots have a real counterpart: the swarm-bot robotic platform 1 14 1. The 
platform consists of a number of mobile autonomous robots called s-bots, which have 
been used for several studies, mainly in swarm intelligence and collective robotics — see 
for instance GroB at al. [15] and Nouyan et al. Ifl6ll . The simulated s-bots are of round 
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Fig. 1. Depiction of (a) the narrow-nest environment used in the first experiment and (b) the wide- 
nest environment used in the second experiment. The gray stripes are the source (left), and the 
nest (right), each 0.25 m deep. The black stripe is the exchange zone, that is 0.5 m deep. The 
light source is marked with "L". 



shape, with a diameter of 0.116 m. Each of them is equipped with 16 infrared proxim- 
ity sensors, used to perceive obstacles up to a distance of 0.15 m. Eight ambient light 
sensors can be used to perceive light gradients up to a distance of 5.0 m. The robots are 
equipped with 4 ground sensors used to perceive nest, source and exchange zone. A 8 
LEDs ring is used to signal when a prey object is carried. An omni-directional camera 
allows the perception of LEDs in a circle of radius 0.6 m surrounding the robot. A uni- 
form noise of 10% is added to all sensor readings at each simulation step. The robots 
can move at a maximum speed of 0.1 m /s by means of a differential drive system. 



4.3 Controller 

All the robots share the same, hand coded, finite state machine controller depicted in 
Fig. 12 The controller consists of two parts, each corresponding to a possible subtask a 
robot can perform. Gray states refer to the harvest subtask, white states to the store 
subtask. Since all the robots start in the harvest area, their controller is initially set to 
perform anti-phototaxis. In this way they will reach the source, where they can start re- 
trieving prey objects. The behavior of each robot is a function of the task it is performing. 
Harvesters not carrying a prey object move towards the source, where they can find prey. 
Harvesters carrying a prey object, move to the exchange zone and wait for a free storer. 
Upon arrival of such a storer, the harvester passes the prey object to it. Storers carry- 
ing a prey object move towards the nest, where they can deposit the object. Storers not 
carrying a prey object head to the exchange zone and search for a harvester with a prey 
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Fig. 2. Simplified state diagram of the controller of the robots. Gray states belong to the harvest 
task, white states to the Store task. The obstacle avoidance state has been omitted for clarity, 
as it is applicable in all states of the robot. t w is the time spent in the exchange zone and is the 
threshold. 

object. Robots can detect other robots carrying a prey on the basis of the color of their 
LED ring. While moving, each robot avoids obstacles (walls and other robots). 

Task switches can occur: a harvester carrying a prey object can decide to become 
a storer, and a storer not carrying a prey object can decide to become a harvester. As 
mentioned before, robots switch task depending on an internal threshold 0, representing 
the maximum amount of control cycles they can spend in the transfer zone waiting to 
pass (harvesters) or receive (storers) a prey object. If a robot remains in the transfer 
zone longer than its threshold without passing or receiving prey objects (t w > 0), it 
switches its task. The optimal threshold value is not trivial to determine. In the work 
presented here, we use a simple method to set the threshold 0: at the beginning of the 
experiment, each robot draws a random threshold, sampled uniformly in the interval 
[0, 1000] . We chose this method because it is independent of the environment and does 
not rely on complex approximation techniques. The threshold value does not change 
during the experiment. In case of the non-partitioned strategy, the threshold is set to 
& = 0, causing the robots to switch subtask immediately as soon as they reach the 
exchange zone. 



4.4 Experiments 

The goal of the experiments is to investigate whether task partitioning can reduce in- 
terference in task-critical zones, and how to allocate a robotic swarm to partitions. As 
pointed out by Lerman and Galstyan [ 1 1, interference is related to the number of indi- 
viduals in the system. Additionally, the physical interference between robots is also a 
function of the environment the robots act in. The higher the group size, the higher the 
density, resulting in a higher amount of physical interference. Thus, in order to study 
interference in our experiments, we increase the size of the group in each of the two 
environments shown in Fig. Q] while using both strategies (non-partitioned and parti- 
tioned). We study the performance of the system when the group size N ranges in the 
interval [1 , 40] . We run 50 repetitions for each value of N and each experimental setting. 
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4.5 Metrics 



In order to quantify the influence of interference, we measure the group performance 
P by the number of prey objects collected by the swarm at the end of the experiment 
(imax = 1 hour). From the group performance measure we can derive the individual 
efficiency as follows: 

I eS = P/N, (1) 

where N is the size of the group. Individual efficiency can help to understand the effect 
of interference on the performance. 

In order to measure the influence of environmental features on the interference, we 
define an interference measure taking inspiration from Rosenfeld et al. |[T7l . In their 
work, interference is measured as the time spent performing actions not strictly related 
to the task, but rather lost due to negative interactions with the environment (e.g., ob- 
stacle avoidance maneuvers). By registering the number of collisions for each area of 
the arena, we can draw conclusions about where physical interferences happen most 
often. We measure interference through the state of the controller: in our case a robot is 
experiencing interference each time its controller perceives an obstacle. 

In case of a partitioned task, there is another source of inefficiency that adds to inter- 
ference: the time lost in the exchange zone. We define the strategy cost C as the sum of 
time lost because of physical interference and time lost in the exchange zone: 

t> = J- hit + -L part > (2) 

where Tj nt is the amount of time steps during which the controller perceives an obstacle, 
and T part is the total amount of time steps spent in prey passing maneuvers. By using this 
metric, the cost of the non-partitioned strategy is purely due to interference (T part = 0), 
while in case of the partitioned strategy, prey passing costs add to interference costs. In 
a way, passing a prey object produces another kind of interference in the system. The 
strategy cost captures this effect, thus allowing for a comparison of strategies. 

5 Results and Discussion 

The graphs in Fig. [3^ and|4^ show the performance P for different group sizes in the 
narrow-nest and wide-nest environment respectively. 

Figure |3p and and[4j) show the individual efficiency I e s of the robots in the narrow- 
nest and wide-nest environment respectively. Black curves are the average computed 
over the 50 repetitions of each setting, gray curves indicate the 95% confidence interval 
on the expected value. The performance graph in Fig.[3k shows that the partitioned strat- 
egy improves performance in the narrow-nest environment. The graph shows that the 
non-partitioned strategy performs better than the partitioned strategy for small group 
sizes (up to N — 13 robots). However, increasing the group size makes the non- 
partitioned strategy collapse: the number of gathered prey objects drops dramatically 
for groups larger than 13. The individual efficiency graph (Fig.|3j3) can explain the be- 
havior of the system. The robots employing the partitioned strategy are less efficient, 
for small group sizes, than those performing the non-partitioned strategy. However, the 
addition of more individuals affects the efficiency of the non-partitioned group in a 
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Fig. 3. (a) Performance P and (b) individual efficiency 7 e g for increasing number of robots in the 
narrow-nest environment. The black continuous line refers to the case of no task partitioning, the 
black dashed line to the case of partitioning. Gray lines indicate the 95% confidence interval on 
the expected value. 



more dramatic way. At a certain point, the drop in efficiency becomes very steep for 
the non-partitioned strategy. On the other hand, the partitioned strategy scales better: 
individual efficiency drops smoothly. This explains why a group using the partitioned 
strategy performs better: it can benefit from the work of more individuals and therefore 
collects more prey objects. These considerations do not hold in the wide-nest environ- 
ment. Figure[4f) shows that the individual efficiency drops smoothly for both strategies. 
In addition, for group sizes N < 33 it is always higher in the non-partitioned strategy. 
Therefore, the non-partitioned strategy performs better than the partitioned strategy, as 
shown by the graph in Fig. [4^. In both the environments, independently of the strat- 
egy used to accomplish the task, the system collapses when the area is saturated by 
the swarm. Figure|5] shows the effect of an increasing number of robots on the strategy 
cost in the narrow-nest environment. The graph compares the cost C of each of the two 
strategies for different group sizes. 
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Fig. 4. (a) Performance P and (b) individual efficiency I e g for increasing number of robots in the 
wide-nest environment. The black continuous line refers to the case of no task partitioning, the 
black dashed line to the case of partitioning. Gray lines indicate the 95% confidence interval on 
the expected value. 



In case of the partitioned strategy (Fig.[5h), the graph shows each component of the 
cost (Ti nt and T part ). Clearly, task partitioning has the effect of reducing the cost due 
to interference but has the disadvantage of increasing the cost due to time lost. The 
probability of two or more robots encountering each other increases with the robot den- 
sity. Although this determines a higher interference cost (i.e., Tint), it decreases the cost 
due to lower waiting time (i.e., T part ) in the case of the partitioned strategy. Partition- 
ing performs better when the gain from interference reduction is greater than the loss of 
performance due to partitioning inefficiencies. These considerations hold in the narrow- 
nest environment, where the likelihood of physical interference in a task-critical zone is 
very high. In the wide-nest environment, interference in the nest is as likely as interfer- 
ence in the exchange zone. Thus, it is not beneficial to pay the cost of waiting and the 
non-partitioned strategy performs better for any group size. The mechanism by which 
partitioning reduces interference costs can be deduced by comparing the interference 
graphs in Fig. [6] The graphs show the number of times that physical interference (as 
defined in Sect. 14.51 ) was registered in each region of the narrow-nest environment. The 
total area was discretized in squares of 1 cm 2 . Figure [6] shows the results obtained with 
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Fig. 5. Cost of interference in the narrow-nest environment. Bars represent the cost C, sum of 
interference time T[ nt and partition time T pa rt (i.e., waiting times). For easy reference, the outline 
of the bars of the respective other graph has been added to each graph, (a) Costs for the partitioned 
strategy, where interference cost stem from waiting times and collisions, (b) Cost in case of the 
non-partitioned strategy, where only physical interference through collisions exists. 

18 robots, in the case of the non-partitioned strategy (Fig. [6^) and in the case of the par- 
titioned strategy (Fig. [6}}). The graphs show that the use of the non-partitioned strategy 
leads to high interference in the nest, which becomes congested. Partitioning the task 
reduces the robot density in the nest, thus spreading the interference more uniformly 
across the arena. In addition, the overall interference diminishes because the exchange 
zone is wider: the robots have more freedom of movement and collide less often. Al- 
though the graphs show only data collected with 18 robots, experiments with different 
group sizes produced similar results. 



6 Conclusions and Future Work 



Interference can be an issue when working with swarms of robots. In this work, we 
used task partitioning and allocation to reduce interference between robots sharing the 
same physical space. We manually partitioned the environment and employed a simple 
self-organized strategy for allocating individuals to subtasks. Results show that a parti- 
tioning strategy improves performance in a constrained environment. Additionally, we 
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Fig. 6. Mean interference values registered for (a) the partitioned strategy and (b) the non- 
partitioned strategy, both in the narrow-nest environment. Shown values are observation means 
of 50 repetitions with N — 18 robots. Coordinates on the x- and y-axis are given in meters. 
The arena is stretched along the y-axis for better visualization. The dashed white line marks the 
location of the exchange zone. 



identified cases in which partitioning is not advantageous and a non-partitioned strategy 
should be used. The proposed strategy is fairly simple and far from being an optimal 
solution, nevertheless we improved the performance of the swarm when interference 
was costly. We believe that our approach can be generalized to a wider range of ap- 
plications. It can be applied to those situations in which system's performance drops 
because of concurrent access to a shared resource. If the accesses can be distributed 
more efficiently by partitioning the resource, the impact of interference can be limited. 
Future work will concern the identification of the optimal allocation in the studied en- 
vironments as well as the development and study of a strategy that can find this optimal 
allocation in a self-organized and adaptive way. In addition, the interference metric pro- 
posed in Sect. 14.51 could be used by the robots to decide whether to partition the task. 
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In this way, we could achieve even better performance, since partitioning would be em- 
ployed only when strictly needed. Finally, the goal is to validate the system using the 
real robots. 
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Abstract. Given an environment and a robot, how much of the environment is 
reachable or accessible to the robot? This fundamental problem in robotics is 
known as reachable workspace estimation and is closely related to the problem 
of determining possible kinematic motions of a robot. For mobile kinematic struc- 
tures with high degrees of freedom (DOFs) in cluttered environments, the motion 
planning problem is known to be NP-hard. Given the intractability of the prob- 
lem, we present an efficient probabilistic method for workspace estimation based 
on the use of a hierarchical strategy and a probabilistic motion planner. The prob- 
abilistic motion planner is used to identify reachable portions of the workspace 
but rather than treating each DOF equally, a hierarchical representation is used to 
maximize the volume of the robot's workspace that is identified as reachable for 
each probe of the environment. Experiments with a simulated mobile manipulator 
demonstrate that the hierarchical approach is an effective alternative to the use of 
an estimation process based on the use of a traditional probabilistic planner. 

Keywords: Reachable workspace, Probabilistic motion planning, Mobile 
robotics. 



1 Introduction 

The reachable workspace W reac h of a robot is defined as the volume or space that 
can be reached by a reference point of the mechanism [8|. Reachable workspace es- 
timation is a fundamental problem in robotics as workspace properties can represent 
important criteria in the evaluation and design of mechanical manipulators [10 3], hu- 
manoid robots [14| and environmental layouts [13]. To take a specific example of the 
task, consider the problem of assessing the accessibility of an environment for a user 
confined to a wheelchair. For an environment to be well suited for wheelchair use it 
should be sufficiently clear of obstacles so that the wheelchair can navigate the environ- 
ment. The process of evaluating an environment for wheelchair accessibility requires 
the detailed involvement of skilled clinicians and many countries provide prescriptive 
mechanisms (e.g., JT)) to guide this process. By reformulating this task as the problem 
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Fig. 1. Establishing accessibility for a robotic wheelchair, (a) The Pond Residence in York Uni- 
versity; (b) The robotic wheelchair 'PlayBot' 1121 : (c) For the Playbot (b) placed in the entrance 
of a two-bedroom apartment in building (a), how much of the environment is reachable? (d-f) Ac- 
cessible regions of the apartment under different furniture layouts. The top views of (d-f) show 
the residence with different placements of furniture within the environment. The bottom views of 
(d-f) show reachable regions (colored green) for the corresponding furniture placements as deter- 
mined by the algorithm presented in this paper.. Note that moving objects around the environment 
can lead to significant changes in the reachable portion of the environment. 



of establishing W re ach for a user confined to a wheelchair it is possible to develop algo- 
rithmic solutions to the problem. Fig.[T]illustrates the problem of estimating the reach- 
able workspace in a student apartment at York University. Student apartments consists 
of a number of fixed pieces of furniture (built in cabinets, etc.) and a number of movable 
components (tables chairs, etc.). Fig.QJa) provides an external view of a student apart- 
ment at York University. Fig.QJb) shows the robotic wheelchair robot (PlayBot |12|) 
with its manipulator. Fig. [TJc) shows the floor plan of the internal student apartment 
with the wheelchair placed at the entrance. Fig.QId)-(g) show the estimated workspace 
Wreach of the robotic wheelchair in the environment under different furniture place- 
ment options. Certain furniture placements restrict the motion of the wheelchair 
considerably. 
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Estimating W reac h for a mobile robot can be expressed in terms of the ability of the 
device to plan motions within its environment. In basic motion planning, given a robot 
A and a static workspace Wcontaining a set of obstacles, the objective is to determine 
a collision-free path between the specified start and goal for A A configuration c of 
A is a specification of the position and orientation of A in W[9]. A configuration c 
is said to be free if A positioned at c does not collide with any obstacles in W. The 
free configuration space Cf ree is defined as the set of all free configurations of A The 
motion planning problem can therefore be formulated in terms of computing a path in 
Cf ree between two given configurations. 

Traditionally the problem of motion planning assumes that each of the degrees of 
freedom (DOFs) are equivalent but in estimating a robot's workspace it is important 
to observe that some DOFs are likely to be more important than others in terms of 
determining how much of the workspace is reachable. In addition, it may be possible 
to construct subspaces of the workspace defined by arbitrary configurations of certain 
combinations of joints and to use these subspaces to carve out large regions of the 
workspace as being reachable for each solution of the motion planning problem. Moti- 
vated by these observations we explore the use of a hierarchical structure of the DOFs 
of the kinematic device to establish the entire Wreach- In this hierarchy we order the 
DOFs of the robot in terms of their predefined "importance". Then we consider cor- 
responding sub-versions of the kinematic structure in which sub-versional joints are 
considered over their range of motion. Each of these sub-versions defines a reachability 
subspace that can be established as being reachable with a single probe into the robot's 
workspace. This hierarchical search mechanism can be used to enhance the probabilis- 
tic algorithms for reachable workspace estimation and to potentially carve out large 
regions of the space with each query within it. 

This paper is structured as follows. Section|2]reviews existing algorithms for reach- 
able workspace estimation. Section [3] outlines our hierarchical approach. Section [4] in- 
cludes comparison results from applying the hierarchical approach and basic PRM to 
a simulated mobile manipulator. Finally Section [5] summarizes the work and provides 
possible directions for future research. 

2 Previous Work 

Although a range of techniques exist for reachable workspace estimation BI6I14I most 
approaches consider the problem for robotic manipulators and do not consider arbi- 
trary obstacles in the environment. Robotic manipulators are fixed at one end and this 
assumption provides certain efficiencies for reachable workspace estimation. For ex- 
ample, one straightforward method to compute Wreach is to take planar sections of the 
workspace defined by the joint angles that make up the kinematic structure and deter- 
mine the contour of the section in the plane. Rotating and translating this plane based 
on other joints in the chain yields the three-dimensional workspace CTT1 . 

In 1 8 ] a numerical approach calculates the exact Wreach by tracing boundary surfaces 
of the workspace. In this approach, an imaginary force is applied to the reference point 
at the end-effector in order to achieve the maximum extension in the direction of the 
applied force. The manipulator reaches its maximum extension when the force's line of 
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action intersects all joint axes of rotational joints and is perpendicular to all joint axes 
of prismatic joints. The drawback of this algorithm is its exponential time complexity 
and that it only deals with manipulators that have ideal joints (without limits). A more 
efficient system was later developed for computing W rea ch for manipulators with joint 
limits [2|. The system decomposed the problem into two subproblems: workspace point 
generation by direct kinematic based techniques and surface computation by extracting 
the workspace contours utilizing a subset of the points generated in the first module. 

Much work has been done on capturing workspace properties for interesting kine- 
matic structures such as human arms 1101141 and reconfigurable robotic arms 0. 
Lenaric and Umek [10] developed a simplified kinematic model to represent human 
arm motion. Wreach was determined by calculating the reference point on the wrist 
for all combinations of values of joint coordinates inside the given ranges. Another ap- 
proach presented by Zacharias et al. [ 14 1 discretizes the workspace into equally sized 
small cubes. Into each cube a sphere is inscribed and sample points on the sphere are 
examined using inverse kinematics. The percentage of the points on the sphere that are 
reachable is used to represent its level of reachability. 

By reformulating the reachable workspace estimation problem in terms of the kine- 
matic planning problem we can leverage results from the robotic motion planning lit- 
erature. Motion planning emerged as a crucial and productive research area in robotics 
in the late 1960's [9| and its applications in real world problems continue to attract 
researchers from all over the world. A complete solution to the motion planning prob- 
lem is known to be exponential to the robot's DOF [4|. As a consequence a number 
of heuristic approaches to path planning have been developed. The main difference 
between the probabilistic approaches and earlier complete approaches is that the proba- 
bilistic approaches do not attempt to construct an exact representation of C/ ree . Rather 
they create a simplified graph that approximately "covers" Cf ree and captures its con- 
nectivity in reasonable time. The Probabilistic Roadmap Method (PRM) [5 7| is a pop- 
ular heuristic motion planner. The algorithm first constructs a roadmap by connecting 
randomly sampled collision free configurations and then answers multiple queries by 
attempting to connect them to the roadmap. 

In this paper we develop a hierarchical probabilistic algorithm based on PRM to give 
a proper estimation for Wreach for an arbitrary mobile device operating in a known and 
cluttered environment. 



3 Hierarchical Reachable Workspace Estimation 

For a robot A moving in the workspace W, the robot's degrees of freedom (DOFs) are 
the minimum set of independent displacements/orientations that specify As complete 
position and orientation in W. Thus a configuration c of A with n DOFs can be specified 
as a set of n parameters, say ji,...,j n , and theoretically there can be 0(n 2 ) different 
orderings of the joints. First we define an ordering of the DOFs such that more important 
DOFs have a lower index. Although there can be many definitions according to the 
nature of the problem, here we define the importance of a DOF by its effect on the 
volume of ,4's occupation in W. This importance weight is expressed more formally 
below. 
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Fig. 2. (a) A consists of Abase and Aarm that has two links L\ and L2 connected by revolute 
joints. The configuration of A is written as (a;, y, 8, <j>i, ^2). (b) Representations of the hierarchi- 
cal occupancy of ^4 are shown in different colors. The body itself also has occupancy constraints 
but these are not shown. 



For a point a in A, let P a : C — > W be the mapping that calculates the position of 
point a in W when A is placed at configuration c. Depending on the position of a in A, 
P a is a function of c' C c. We say that a is determined by a DOF jj if ji G c'. Now we 
can define a weight function of a DOF ji in terms of the volume of A in W determined 
by it: 

Defn 1: w(ji) =\ {a G A \ a is determined by ji} \. 

A DOF j x is more important than j y if w{j x ) > w(j y ). Therefore we can write the 
configuration c as a vector of length n in a decreasing order of their importance, say 
c= {jx, j2, ■■■jn), i-e. for i = 1,2, ...n— l,w{ji) > w (ji+i). Take the 5 -DOF mobile 
manipulator shown in Figure|2]as an example, x and y determines the entire robot, and 
9 determines all the portions except the rotation center of the mobile base (assume the 
mobile robot can rotate around its center). <f>x determines links L\ and L^. (f>2 deter- 
mines link L-2- Therefore, w(x) ~ w(y) > w{9) > w(<fix) > w{(j>2), and the ordered 
configuration can be written as {x, y, 9, <t>x,4>2) (or (y, x, 9, <f>x, 4>2))- 

Given an ordering of the DOFs, we seek a hierarchical representation within which 
certain joint angles are 'free' and can assume arbitrary values within some previously 
defined domain. Let the domain of ji be T>i,c r — (jx>J2> ■■■jr) is a subset of £>i xT>2X 
. ..£>„, given by {Vx r+ i G T> r+ x,x r +2 € T> r+2 ,...x„ G V n | {jx,J2,-jr,Xr+i,%r+2, 
...£„)}. That is c r is the set of possible configurations with joints l...r having spe- 
cific values but joints r + l...n being free. This hierarchical concept applies to general 
kinematic structures in the domain of motion planning. 

In order to integrate the above hierarchy into the workspace estimation process, we 
first perform two types of analysis to the hierarchical representations: occupation anal- 
ysis and reachability analysis. Define the occupied area OA c (c r ) of c r as the union of 
the occupied volume in W of every element in c r , and the reachable area RA C (c r ) of c r 
as the union of the reachable points of every element in c r . Under the hierarchy nodes 
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with lower r occupy and reach larger workspace than those with higher r. To be precise, 
we have these two lemmas: 

Lemma 1: Vi,j e [0,n],i < j =>• OA c (c % ) D OA c (c>). For any configuration 
of A, the occupied workspace of the lower hierarchy is the superset of that of the higher 
hierarchy. 

Lemma 2:\/i,j G [0,n],i < j =>■ RA c (c l ) D i?A c (c J ). For any configuration of 
A, the reachable workspace of the lower hierarchy is a superset of that of the higher 
hierarchy. 

In addition, let V(c) be a function that returns true iff A is collision free when it is 
at configuration c. Similarly, V(c r ) returns true iff every element in c r is valid, i.e. 
OA c (c r ) does not collide with any obstacle in the environment. Therefore, we have the 
following lemma: 

Lemma 3: \/i,j s [0, n],i < ] 'A V{c l ) =>■ V(cJ). For some configuration of A 
that its lower hierarchical representation is free implies the higher hierarchical repre- 
sentation is free, too. 

The spatial representation of both the reachability and occupancy spaces can be very 
complex. In practice the computation of the exact hierarchical representations is unnec- 
essary. Conservative representations of these complex shapes can provide significant 
computational savings and this computation can be done prior to the execution of the 
motion planner. This needs to be done only once for each DOF of the robot, independent 
of the robot's configuration. 

To establish the general representation of OA(c r ) and RA(c r ), we fix the first r 
DOFs and take all possible values of the remaining DOFs to construct the hierarchical 
body. Note that RA(c r ) defines a subspace in the robot's workspace that is reachable by 
some unique configuration within c r , and that for any two configurations within c r there 
exists a continuous path between them. As an example, consider the mobile manipulator 
A shown in Fig. EJa), let V x = [x min ,X max ], T> 2 = [Vmin, Umax], 1^3 = [-7r,7r), 
Z?4 = [— 7r, tt), and T>$ = [— tt, tt). Fig|2b) shows the hierarchical representation of A 
from level 5 down to 2, indicated by color. For simplicity we used ideal ranges [—71", tt) 
for the rotational joints of A 

The hierarchy can be integrated into workspace estimation using a probabilistic plan- 
ner such as PRM. We now describe the main steps involved in the construction of a 
hierarchical roadmap 1Z for efficient reachable workspace estimation. Nodes with large 
reachable areas are preferred (they establish more of the environment as being reach- 
able for each calculation). So for each configuration c, we look for the minimum value 
r m i n such that v(c Tmi ") is true, and we call r m i n the rank of c. The procedure de- 
scribed in the pseudocode in Algorithm 1 finds a random configuration and establishes 
its most general representation in the hierarchy. In the "for" loop from Line 4 to Line 
9, the algorithm computes the rank of the node by checking collisions of the hierarchi- 
cal representations of the robot's occupancy. Once the minimal valid hierarchical node 
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(a) Classification of a node 




# nodes = 1 



# nodes = 50 



# nodes = 500 



(b) Hierarchical PRM in action 

Fig. 3. Construction of Tl. Upper row shows the generation and classification of one node. The 
rank of a node c r is calculated by checking the hierarchical occupancy representation of A The 
lower row shows the hierarchical PRM in operation. The coverage and connectivity of TZ in- 
creases as more nodes are added. Node color encodes the classification of the node. 



is established the configuration together with the computed rank is added to the set of 
nodes N (Line 11). 



Algorithm 1: Node selection 



1: nodeFound <— false 

2: while -^nodeFound do 

3: c <— a randomly chosen configuration in C 

4: for k «— 1 to n do 

5: ifV(c fc )then 

6: nodeFound *— true 

7: break 

8: end if 

9: end for 
10: end while 
11: 7V^iVu{c fe } 
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Whenever a new hierarchical node is found, we select a number of candidate nodes 
from the current set N and try to connect the new node to each of them. In addition 
to the connection computation performed by the traditional local planner, the rank of 
the edge should be established. For an edge e we look for the minimum hierarchy r m i„ 
such that V(e Tmin ) is true along the edge. 

The hierarchical node interconnection is built upon a local path locator and a hierar- 
chy establisher. The local path locator returns an edge candidate, i.e. a local path that A 
can follow from one configuration to another. Then the hierarchy establisher checks if 
the edge candidate is collision free and meanwhile establishes the edge's most general 
representation in the hierarchy. The process of establishing hierarchical node intercon- 
nections is outlined in Algorithm 2. In line 3, the hierarchy r is initialized to be the 
maximum value of the ends of the edge. There is an obvious lemma according to the 
definition of the hierarchical edge connecting two nodes a Tl and b T2 : 

Algorithm 2: ConnectfcT 1 , b n -) 

1: r <— the edge candidate returned by the local path locator 

2: Discretize r into a list of configurations r' = (ci , C2 , . . . , c m ) 

3: r curr ent <— MAX(n,ra) 

4: for all a 6 r' do 
5: for k <— r current to n do 
6: ifV(e*)then 

'• ' current * K 

8: break 

9: else 

10: exit and report failure 

11: end if 

12: end for 

13: end for 

14: B^BU{(a,i)'' c ™ t } 

Lemma 4: V(e r ) = true =>• r > r\ A r > r<i, i.e. the rank of an edge is not less than 
the rank of either end node of the edge. 

Algorithm 2 searches over the sequence of configurations on the edge for verification 
and hierarchy establishment. This general approach is straightforward to implement. 
Note that the hierarchy is established through the validation of the configurations. Sim- 
ilarly, to apply this strategy to other probabilistic motion planners it can be done when 
samples are checked for collision. 

After the hierarchical roadmap 1Z is built, W re ach can be computed from the con- 
nected component of 1Z that contains the initial configuration of A through the mapping 
function RA. Uniform cell decomposition can be used to represent W. Fig.|4]shows the 
estimated W re ach of a 3D environment. 

4 Experimental Validation 

We conducted experiments of our algorithm using the simulated 5-DOF mobile ma- 
nipulator A shown in Fig. [2] First we provide an example that illustrates the algorithm 



Exploiting Hierarchical Probabilistic Motion Planning 237 




Top view 1 : T =122 
Top view 8: T= 

Top view 3: T=40 




(a) 3D workspace view of the mobile robot in a room 




Top view 1 Top view 2 

(b) Slices through the workspace 



Top view 3 



Fig. 4. Given a mobile robot with 5 DOF entering an apartment-like environment, the estimate of 
the 3D reachable workspace W re ach (green area) shown in different horizontal layers 



in operation (see Fig. [3}. Initially 1Z contains only one node that represents the initial 
configuration of A The rank of each randomly generated node is determined by look- 
ing for the most general occupancy representation of A that does not collide with any 
obstacles. Similarly the rank of each edge is determined by looking for the most general 
occupancy representation of A along the edge that does not collide with any obstacles. 
Fig-H a ) shows tests for a randomly generated node, c 2 and c 3 generate collisions while 
c 4 does not, so this specific node is classified as c 4 . The construction of 1Z is incremen- 
tal. Fig. [2b) shows incremental changes in 1Z as more nodes are added to the graph. As 
more nodes are added both coverage and connectivity of 1Z increases. 

Fig. [6[a) and (b) compare the hierarchical PRM and random sampling using repeti- 
tive PRJVI for reachable workspace estimation on the environments shown in Fig. [2a) 
and (b) respectively. The x-axis represents the number of samples used in the estima- 
tion. The y-axis represents the percentage of the estimated W re ach (the true W re ach 
of the robot was computed by brute force search). Because randomness is involved, 
the workspace volume for ten independent runs for each case were averaged. Standard 
deviations are also plotted. Both graphs show that the hierarchical approach is more 
effective than repetitive PRM sampling for reachable workspace estimation. 
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(a) 



<b) 



Fig. 5. Experimental setup, (a) The mobile manipulator is placed in a simple 3D environment; (b) 
The mobile manipulator is placed in an apartment-like environment. 
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(a) Experimental result for the environment 
shown in Fig.[5la). 
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(b) Experimental result for the environment 
shown in Fig.|5jb). 



Fig. 6. Comparisons of workspace volume computed from hierarchical PRM and repetitive PRM 
sampling for the environment shown in Fig.[5fa) and (b). Each data point plots the average of ten 
experiments. Standard deviations are plotted. 



Finally we evaluate the time efficiency of our hierarchical approach. Fig.|7]compares 
the running time of the roadmap construction using the basic PRM and the hierarchi- 
cal approach. Because randomness is involved, running times for 20 independent runs 
were averaged for each case. The hierarchical PRM performed reasonably well in these 
experiments. As can be seen from the results, creating a hierarchical roadmap takes less 
time than creating a traditional roadmap. This is because the hierarchical PRM saves 
time in collision checking in easy regions. 
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Roadmap Construction (PRM v.s. HPRM) 
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Fig. 7. Comparison of the average running time of the construction of traditional roadmaps and 
the hierarchical ones for the model given in Fig. Ob). Averages are for 20 trials. Standard devia- 
tions are shown. 



5 Summary and Future Work 



A hierarchical approach was presented for adapting probabilistic motion planners for 
reachable workspace estimation. Unlike traditional probabilistic motion planners that 
treat each DOF equally, we order the DOFs of the kinematic structure and consider a 
hierarchical approach to the planning task. Considering the characteristic of the reach- 
able workspace estimation problem, this hierarchy exploration improves the planning 
process through two critical computations: occupational analysis and reachability anal- 
ysis. Validation of configurations begins by doing fast tests on simple occupational 
representations and only progresses to more accurate (and more expensive) evaluations 
as necessary. Because randomness is involved it is hardly possible to estimate the entire 
reachable workspace from the probabilistic roadmap within reasonable time. However, 
by iteratively computing the maximal reachable workspace from each node and edge 
our hierarchical motion planner can be more effective in the computation process than 
the traditional ones. 

The hierarchical workspace estimation algorithm is especially useful for mobile 
robots in environments with obstacles. Experiments were conducted on a simulated 
5-DOF mobile manipulator in two 3D environments. Experiments show that the hier- 
archical approach can be an effective and efficient alternative to the repetitive PRM for 
reachable workspace estimation. 

Our current hierarchical algorithm uses the coarse-to-fine hierarchical nature in the 
process of estimating the workspace. In the examples shown in this paper, we assumed 
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the DOFs of the robot are ordered in terms of their predefined "importance", which is 
determined by its effect on the volume of the kinematic structure's occupancy in the 
workspace. When the kinematic structure executes a path from the start to goal, it only 
moves its 'less important' DOF (e.g. the attached manipulator is less important than 
the mobile base) when necessary. This reduces unnecessary flailing of less important 
DOFs (e.g. the attached manipulator) while the mobile base is moving. Motivated by 
this feature of the hierarchical approach, our ongoing work includes the use the hierar- 
chical space to do path planning after the space is constructed. The goal is to address 
practical problems where paths should not only meet hard constraints that absolutely 
must be satisfied (e.g. that the path is collision-free) but meet certain soft constraints 
that serve as guides to 'encourage' or 'influence' the robot to behave in certain ways. 
While hard constraints are satisfied during the construction of the hierarchical space, 
soft constraints can be considered by planning within the constructed space. 

For a complex robotic platform the ordering of the DOFs may not be obvious, e.g. 
a mobile robot with two manipulators attached and a more adaptive definition of the 
ordering may be required. A hierarchical characteristic might also be employed in other 
aspects of motion planners. For example, one heuristic would be to let the established 
hierarchy lead the sampling process toward the boundaries of obstacles, i.e. to sample 
more densely near nodes with higher hierarchy labels than those with lower hierarchy 
labels. 

We can also imagine a more sophisticated definition of reachable workspace which 
might involve establishing the number of configurations from which the kinematic 
structure can reach a given location. This might provide insights into different levels 
of reachability. A space for where there exists many reachable configurations should 
probably be considered more reachable than one with just a few. 

Acknowledgements. The financial support of NSERC Canada is gratefully acknowl- 
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Abstract. A new tracking algorithm is proposed for threedimensional object 
tracking. It is based on an active contours algorithm with region-based minimiza- 
tion for twodimensional tracking. The algorithm is extended to use the sharpness 
of the objects enclosed by the contour to determine an estimate for the z position. 
This depth-from-defocus approach is enabled through different sharpness calcu- 
lations, including the grey level variance and column-wise and row-wise variance 
calculation, which allow to analyze the astigmatism. By this, the ambiguity of the 
out-of-focus displacement is resolved. The algorithm can deliver estimates for the 
z position of an object if the object is in a certain range around the focal point, 
without relying on external assumptions. The characteristic of the defocusing is 
determined by executing a focus sweep and calculating the defocus curve for the 
object of interest. The ability of the algorithm to generate threedimensional po- 
sition coordinates is evaluated in a setup with nanopositioners and the principal 
feasibility is shown. 

Keywords: Image processing, Object tracking, Depth estimation, Focus analysis. 



1 Introduction 

The SEM has been widely used as a imaging tool for the automated handling of micro- 
and nanoscale objects (see e.g. [T)). While there have been many manipulations ex- 
ecuted manually, and with the necessary experience these manipulations have a high 
success rate, fully automated handling and manipulation of micro- or even nanoscale 
objects in the SEM still is very rarely encountered. The problems which have to be 
solved are manifold and not easy to handle. Integrated sensors in the actuators or the 
setup may deliver some information about the positions of end-effectors and tools, and 
enable estimates about the position of objects to be manipulated. The real actual po- 
sition information of objects or tools though is difficult to recover from this data, be- 
cause of various factors like thermal drift, play or object interactions on the nanoscale. 
A specific requirement for non-teleoperated processes is therefore the need for sensor 
feedback generation based on SEM images. The SEM is the only sensor which may de- 
liver an overview over the whole scene, enabling the extraction of the positions of most 
or all objects engaged in the manipulation. Only the SEM determines these positions 
in a common coordinate system, which makes it possible to evaluate relative distances 
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Fig. 1. A screenshot showing two objects having different z-positions. One object is a silicon 
nanowire, the other a deformed STM tip. The distance between the objects in z-direction is not 
immediately visible. 

between the objects. One problem which is crucial for the success of any automation 
approach is the missing information about the position of objects in z-direction, which 
means orthogonal to the image plane. An example image illustrating the problem is 
shown in figure Q] 

2 State of the Art 

Different algorithms have been described in the literature extracting the twodimen- 
sional position of objects from SEM images for automation purposes. The performance 
of these algorithms is good, making first simple automation scenarios possible. The 
approaches used for twodimensional tracking on SEM images and their possible exten- 
sions to 3D-tracking will be summarized in the following. 

One of the first and most simple approaches used template matching as the basis of 
the algorithm [2]. A template image is extracted or loaded which contains the object to 
be tracked. The template image is then cross-correlated with a search area in the input 
image. The maximum value of the resulting array is in the place where the template is 
most likely to be found. Due to the use of cross correlation, this approach is very robust 
against additive noise, which is an advantage especially for fast scanned SEM images. 
Problems of this approach are that the algorithm is sensitive against certain changes in 
the object appearance which may occur during handling processes. Examples for these 
appearance changes are rotation of the object, scaling of the object due to magnification 
changes and partial occlusion by other objects in the setup. Removing these weaknesses 
for this method comes with increased computational effort so that a fast enough calcu- 
lation is not always possible. Extraction of the z-position is not featured and cannot 
easily be added. 

If instead of a template image a CAD model of the object is available, it is possible 
to use rigid body models to track the object in the SEM image [ 3 ] . The implementation 
uses measurement lines orthogonal to the model edges and tries to fit the model to vis- 
ible edges in the image. Model edges which should be invisible are identified and not 
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used for the pose estimation. Though edge detection is difficult in noisy SEM images, 
the approach yields good results using advanced techniques for discarding or outweight- 
ing false edges and through the high number of measurement lines used. When three- 
dimensional CAD-models are used, it is possible to recover the threedimensional pose 
including in-plane and out-of-plane rotations, except the z-position. The extension for 
true threedimensional tracking relies on a model of the SEM image projection to yield 
the z-component of the position. This seems to be working for low magnifications. 

Another possibility is the use of active contours or snakes (for details about this 
concept see (4|), which do not rely on much pre-existing knowledge about the object. 
Active contours are parametrized curves in twodimensional space, that means in the 
image plane. After coarse initialization the contour is evolving to segment the object 
from the scene. The contours are coupled with an energy function dependent on their 
shape or appearance, and on the image data. This energy function is being minimized 
by moving contour points or the contour as a whole. The part dependent on the contour 
is called internal energy, the part dependent on the image data is the external energy. In 
the original formulation, the external energy function was defined to be dependent on 
the distance of the contour from edges in the image, as explained in [ 5 1 . For the use with 
noisy SEM images, a region-based approach (see |6J and [71) has shown to be useful. 
The external energy function here is dependent on the region statistics and the noise 
characteristics of the imaging source. The goal is to maximize the compound probability 
of the enclosed region. This approach has proven to be very robust to additive noise, and 
is inherently robust against scaling and rotation. If the contour minimization is restricted 
to the euclidean transform space, robustness against partial occlusion is added. Due to 
the model-free nature of this approach, threedimensional tracking is not immediately 
possible, but the coupling with focus-based methods is principally possible and shows 
first promising results in the SEM. 

In this paper, the last tracking approach is taken as a basis, and extended to use defocus 
analysis for depth estimation. The extracted information is only the z-position of the 
tracked object, without any structural information about the object. For the recovery of 
threedimensional structure of objects, different methods may be used (see e.g. O or (9)). 

3 Principle 

The principle of the twodimensional tracking has been explained already in |6|. The 
important aspect is that the active contour algorithm does not only deliver the position 
information of the object, but at the same time calculates a segmentation of the object 
from the rest of the scene. This enables further analysis of the enclosed object. 

Due to the working principle of the SEM, only a certain range around the set work- 
ing distance is depicted sharp. Though this range is quite big in comparison to optical 
microscopes, defocusing is still evident, as can be seen in figure [2] The defocusing in 
the SEM has been used already in ifTOl to determine the z-position of objects by gen- 
erating a sharpness curve over a certain working distance range. One drawback is the 
amount of time needed to obtain this image sequence and therefore also the disability 
to monitor dynamic processes. But by analyzing the sharpness of the object which is of 
interest, it is also possible to directly conclude to the out-of-focus displacement of the 
object, as has been recently demonstrated for the SEM in [IX]. 
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Fig. 2. Different sharpness states of objects. Top left: sharp object, top right: blurry object, bottom: 
influence of astigmatism on object blur. 



The sharpness measure used here is the grey-level variance 
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and C the contour, A(C) the enclosed area and / the image. 
For the object enclosed by the contour this means 

a 2 (I{WD),A{C)) = Max => z(Object) = WD 



(3) 



with the working distance WD and the z-position of the Object enclosed by the contour 
z(Object). 

Also, assuming a calibration curve 



Cal(Az) = a 2 {I{Az),A{C)) 



(4) 



is available which is monotonic, bijective and describes the relation between out-of- 
focus displacement and sharpness, we can calculate the out of focus displacement of 
the object from the object sharpness for each frame: 



Az = Car\a 2 (I,A(C))) 



(5) 



Defocus-Based Threedimensional Tracking in SEM Images 247 

One problem which still persists after this analysis is that the out-of-focus displacement 
turns out to be ambiguous. Two naive possibilities exist for the solution of this problem, 
one assuming that the object is nearer than the working distance, the other assuming 
that the object is further away. This is a situation which is not optimal for automation 
purposes. Though in some cases the correct solution may be determined by the setup 
and context knowledge, it is desirable to determine the solution without additional in- 
formation apart from the image. For the solution of this problem, a typically undesired 
effect in SEM imaging may be taken advantage of. 

During normal use of the SEM, astigmatism is something that should be diminished 
or removed by astigmatism correction. Astigmatism in the SEM leads to blurry images 
for the user. An important property of astigmatism is that the sharpness is direction 
dependent. The focal points of the electron beam are different for two perpendicular 
directions, as can be seen in figure [2] If we name the two perpendicular directions wo 
and wi: 

WD W0 ^ WD W1 (6) 

To take advantage of this, we additionally to the grey level variance of the image calcu- 
late the grey level variance of the rows and columns separately by using 

al{I,A{C)) = Y, N l (v) E m-I{A{C),y) (7) 



with 
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and C the contour, A(C) the enclosed area and / the image. 

The two values are normalized to their maximum, which is determined during the 
initialization phase (see section^: 

al{I,A(C)) 
a 2 Jl 1 A(C)) = V ,\, T \ L,, (11) 

and 

* 2 M,A(C))= "f^ ^ d2) 

max{a£{I,A(C))) 

In this case we expect a working distance sweep to generate two slightly displaced 
curves like depicted on the left in figure [3] under the assumption that the object in the 
image has suitable structure. 
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Fig. 3. Expected and measured curves for the different sharpness measures 



With the two curves having slightly displaced maxima, we can estimate from the 
ratio of the two values 

_al(I,A(C)) 



to which side the out-of- focus displacement occurs: 

r < 1 => Az < 
r > 1 => Az > 



(13) 



(14) 



4 The Algorithm 



The twodimensional base algorithm is similar to the active contour algorithm described 
in section 12 The principle is depicted in figure [4] Important is that after an initial free 
minimization, the tracker does only translate and rotate the contour as a whole. This is 
necessary to enable the out-of-focus estimator to work, because else the enclosed area 
may change. The same is valid for scaling, which is therefore disabled in the algorithm. 

In order to enable continuous threedimensional tracking, the algorithm has to be 
initialized like shown in figure [5] An initial working distance sweep is carried out to 
acquire the characteristic curve of the tracked object. During this sweep, the twodimen- 
sional tracking has to be enabled already. After this, certain calculations and optimiza- 
tions are carried out on the acquired curves, e.g. it is made sure that the curves are 
monotonic in sections. After the data has been processed, the tracker can track contin- 
uously until either object changes or imaging changes require a reinitialization. 

The threedimensional tracking itself consists of the twodimensional tracking algo- 
rithm, augmented with a sharpness calculation component, an out-of-focus displace- 
ment estimator and a direction estimator, like shown in figure [6] The active contour 
tracker delivers the twodimensional position and the segmentation of the object. This 
segmentation is then used to mask the original image. From the masked image, the 
object sharpness is calculated using variance calculation and the directional sharpness 
measures mentioned in the last section. 

The variance value is used to estimate the out-of-focus displacement by comparison 
with the data acquired during initialization. 

The directional sharpness values are used to estimate on which sidelobe of the ini- 
tially acquired sharpness curve the object is in the actual image. After this is known, the 
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Fig. 5. The initialization steps for the threedimensional tracking 



information about the displacement value and the displacement direction is combined 
with the working distance at which the current image was captured. The result is the 
estimated working distance at which the object is placed. This is then joined with the 
position information from the twodimensional tracking to generate a complete coordi- 
nate set. 



5 Experiments 



For the evaluation of the algorithm, the performance has been tested in a setup inside the 
SEM. The first target object was a chessboard pattern as seen left in figured] This chess- 
board pattern has been put on a stub which is mounted to a XYZ positioning setup using 
piezo slip-stick actuated axes with internal position sensors. For the experiment, the in- 
ternal sensors of the axes were read out and the acquired data joined with the tracking 
data. During the experiment the axes were moved in closed loop mode using the internal 
sensors. The movement was in a pattern to verify the algorithm performance. All mea- 
surements have been executed at a magnification of 800x and a scanspeed of 5 (frame 
averaging of 2 5 frames) on a LEO scanning electron microscope. The twodimensional 
pattern of the movement has a size of 21/im times 39/im. The pattern has been repeated 
at the z-axis positions of 0.17mm, 0.256mm and 0.32mm. 
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Fig. 6. The complete 3D tracking algorithm in the initialized state 

For a second experiment, the setup seen in figure[7]is used. Here, a gripper is attached 
to a positioning setup with piezo slip-stick actuated axes. Similarly, the internal sensors 
of the axes were used to obtain the reference data. 

6 Results 



The first result was the generation of the expected curves from left in figure [3] The 
result can be seen on the right in figure [3] The shift of the variance calculated in rows 
and columns separately is clearly visible, which enables the algorithm to work in the 
anticipated way, estimating the direction of the defocusing. 

As can be seen in figure [10] left, the movement pattern used incorporated movement 
along each axis as well as in diagonal. Also visible is a slight distortion of the shape 
in comparison to the desired shape. The reason of this may be a slightly decalibrated 
sensor, according to further investigation. Therefore the positions are not perfectly set. 

In figure[9]this gets more obvious. While the tracking x position in figure|9]relatively 
closely follows the sensor information, this is not the case for the y position. Apart 
from the slightly decalibrated sensor it can be stated that the twodimensional tracking 
is working and functional, and the dependence of the measured position on the set 
position is roughly linear. 

The threedimensional component of the position is shown in figure [10] right over 
the x-position. Visible is a slight dependence of the measured z-position from the x- 
position. One factor in this may be the orientation of the setup, but due to the fact 
that the dependence is not equal at different z-positions, it may be concluded that this 
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Fig. 7. The manipulation setup used inside the SEM 




Fig. 8. The structures used for the experiment, a chessboard pattern mounted on a XYZ position- 
ing setup, and a gripper attached to the same setup. Visible is the active contour tracking and 
segmenting one chessboard block and part of the gripper respectively. 



effect could be generated by image characteristics changing slightly when the position 
is changed. This cannot be easily avoided, due to the complex setup and the principle 
of SEM image formation. 

Figure Q~T] shows the determined z-Position from the tracking algorithm. Visible is 
that there is a certain systematic error in the tracked position, as the calculated values 
deviate from the expected values. The set working distance of the SEM is in the middle 
of the working distance range shown in figure[TT] at the value 0.28. The problem which 
occurs here is that the algorithm is most accurate not in the point of maximal focus, 
but within a certain range on the sidelobes of the sharpness curve. As can be seen in 
figure[3] the sharpness curve is relatively flat around the maximum value. In this interval 
around the maximum, small changes in detected sharpness, which may also occur due 
to time variant behavior of the SEM imaging process or due to certain changes in the 
surrounding setup, will result in large errors in the estimated out-of-focus displacement. 
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Fig. 9. The tracked x-position and y-position over the set x-position and y-position 
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Fig. 10. Tracked X-Y position of the chessboard pattern in the repeated measurement 
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Fig. 11. The tracked z-position over time and the 3D trajectory 



This explains also not only the deviation from the axis set value, but also the large 
amount of variation during the movement on the same z-position. So it has to be stated 
that the optimal working condition for this algorithm is a slightly defocused image. 

In figure Qj] on the right, the tracking result can be seen in three dimensions. The 
movement pattern is qualitatively visible, though the tracking in z-direction is not as 
good as in the image plane. Still the goal of the algorithm design has been reached, 
an estimate has been calculated for the z-position of the object which is principally 
useable. 

In figure [T2l the result of the tracking of the gripper is shown. Clearly visible 
here is that the structure tracked on the gripper may not be perfectly suited for the 
3D-tracking. The chessboard in comparison has a certain variance in any direction, 
while here the structure tracked does have more direction dependence. Additionally, 
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Fig. 12. The set and measured z-positions of the gripper during two tracking sessions 

the structure contributing to the sharpness calculation is mostly the holes in the gripper, 
which are relatively few and big compared to the tracked area. This leads to suboptimal 
measurement curves, despite the errors which occur in both curves, it can be stated that 
threedimensional tracking is possible and working. 



7 Summary 

In this paper a threedimensional tracking algorithm for the tracking of objects in the 
SEM has been presented. The algorithm takes advantage of the image defocusing which 
is evident when objects leave the focal plane. A twodimensional tracking algorithm 
based on active contours with a region-based minimization has been taken as the base 
algorithm. Added extensions include the segmentation of the object and the consecutive 
sharpness calculation. Additionally the variance of the rows and columns is calculated 
separately for determining the direction of the defocusing. This enables the analysis of 
the sharpness in different directions. If the image contains astigmatism, it is possible 
to estimate if the image focal plane lies in front of or behind the object. Experiments 
have shown that this approach is working and after an initialization phase qualitatively 
delivers a threedimensional position information. The z-position still contains a sys- 
tematic error, which is most influential around the best focused point. This error has to 
be diminished by further analysis and change and optimization of the implementation. 
Overall the feasibility of this threedimensional tracking algorithm has been shown. 
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Abstract. In this paper we present and evaluate several methods for real-time 
environment representation by extracting object delimiters from the traffic 
scenes using a Dense Stereovision System. The delimiters detection is based on 
processing the information provided by a 3D classified occupancy grid obtained 
from the raw dense stereo information. One of the problems in representing the 
environment through the occupancy grid is a large volume of data. Therefore 
we propose a more compact 2.5D model by representing the environment as a 
set of polylines with associated features. Two approaches to extract object 
delimiters are presented: an improved contour tracing called 3A Tracing and a 
polyline extraction method through the radial scanning of the occupancy grid. 
We discuss the advantages and drawbacks for each of these methods. 

Keywords: Environment Representation, Contour Tracing, Border Scanning, 
Environment Perception, Polyline Extraction, Object Delimiters. 



1 Introduction 

In the context of in-vehicle navigation systems, the environment perception and its 
convenient representation is an important requirement [1]. The process of 
environment representation building has to be accurate and characterized by a low 
computational cost. 

Usually, the Driving Assistance Applications detect the objects through 2D or 3D 
points grouping processes. The detected objects are represented by geometric 
primitives such as 2D bounding boxes [2] or 3D cuboids [3]. As an alternative 
approach, the objects may be represented by polylines. One of the advantages of the 
polyline based objects representation is the close approximation of the object contour 
by the polygonal model while having a number of vertices as small as possible. In the 
same time the polyline could inherit the type, position and height properties of the 
associated object. 

The polyline object representation may lead to the creation of subsequent 
algorithms that are computationally fast due to the fact that only a small subset of 
points is employed. 

The road feature identification through the object delimiters detection can be used 
in the unstructured environments as an alternative solution to the lane detection 
algorithms. 
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The object delimiters extraction is studied in some areas like mobile robots [4], [5], 
[6], [7], [8] or autonomous vehicle systems [9], [10], [11]. The polyline representation 
is very common in many algorithms, such as localization and mapping [6], [7], [8], 
[10] contour tracking [12] and path planning [10]. 

The polyline extraction methods differ by the nature of the information as well as 
by the sensors used for data acquisition process. Current systems use laser [4], [8], 
[9], [10], sonar [11], [7] or vision sensors [11]. 

Two main directions can be distinguished for the delimiters extraction: 

■ The contour processing of already detected objects from the scene [13]; 

■ The radial scanning of the environment. This method is common for the 
systems based on sonar or laser sensors [4], [9]. 

A method for map representation as a set of line segments or polylines is described in 
[7]. An occupancy grid is created here from sonar information. The data is converted 
to a list of vertices using the Douglas Peucker line reduction algorithm. 

In [8] a method that learns sets of polylines from laser range information is 
presented. The polylines are iteratively optimized using the Bayesian Information 
Criterion. 

The polyline representation was chosen in [10] for terrain-aided localization of 
autonomous vehicle. The new range data obtained from the sensor are integrated into 
the polyline map by attaching line segments to the end of the polyline as the vehicle 
moves gradually along the tunnel. 

In this paper we present and evaluate several methods for real-time environment 
representation by extracting object delimiters from the traffic scenes using a Dense 
Stereovision System [3], The delimiters detection is based on processing the 
information provided by a 3D classified occupancy grid obtained from the raw dense 
stereo information. One of the problems in representing the environment through the 
occupancy grid is a large volume of data. Therefore we propose a more compact 2.5D 
model by representing the environment as a set of polylines with associated height 
features. We present two approaches to extract object delimiters: 

■ The 3A Tracing. The classical algorithm for contour tracing is improved by 
developing a new method named 3A Tracing Algorithm; 

■ The radial scanning of the occupancy grid. We have developed a Border 
Scanning method that is able to detect delimiters of complex objects taking into 
account the nature of information from the traffic scene (curb, object, and road). 

A polyline map is generated as the result of the delimiters extraction process. Each 
polyline element inherits the type (object, curb), position and height properties of the 
associated objects from the occupancy grid. 

In the next section, we describe the proposed Delimiters Extraction architecture. 
The delimiters detection approaches are presented in section 3. Experimental results 
are given in section 4, and section 5 concludes the paper with final remarks. 
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2 System Architecture 

Our delimiters detection approaches have been conceived for an urban driving 
assistance system. We extended our Dense Stereo-Based Object Recognition System 
(DESBOR) by developing an Object Delimiters Detection component. A detailed 
description about the DESBOR system is presented in [3]. 

The Object Delimiters Detection system architecture consists in the following 
modules (see Fig. 1): 



Left Camera 



Right Camera 





TYZX Hardware 
Stereo Machine 



Reconstructed 3D Points 



Occupancy Grid Computation 



Object Delimiters Detection 



Output 



Fig. 1. System Architecture 



TYZX Hardware Stereo Machine. 

"TYZX" hardware board [14]. 



The 3D reconstruction is performed by the 



Reconstructed 3D Points. The reconstructed 3D points are used for the occupancy 
grid generation. 

Occupancy Grid Computation. The occupancy grid (see Fig. 2.c) represents a 
description of the scene, computed from the raw dense stereo information represented 
as a digital elevation map (see Fig. 2.b). The occupancy grid cells are classified into 
road, traffic isle and object cells. A detailed description about the occupancy grid 
computation is presented in [15]. 

Object Delimiters Detection. The Object Delimiters detection uses the occupancy 
grid results as the input and generates a set of unstructured polygons approximated 
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with the objects contour. The delimiters can be extracted from the occupancy grid 
through both 3A Tracing and Border Scanning algorithms. 




Fig. 2. The Occupancy Grid (c) is computed from the Elevation Map (b) of a scene (a). The 
occupancy grid cells are roughly classified (blue - road, yellow - traffic isle, red - obstacles). 

Object Delimiters Detection Output. A polyline map is generated as the result of 
delimiters extraction process. For each polyline element we keep the following 
information: a list of vertices, the delimiter type (object, curb), and the height of the 
object for which we apply the polyline extraction. 
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Fig. 3. The car coordinate system 

It must be noted that the car coordinate system coincide with the world coordinate 
system having its origin on the ground in front of the car (see Fig. 3). The position 
and orientation of the stereo cameras are determined by the absolute extrinsic 
parameters [16]. 

3 Object Delimiters Extraction Methods 



A set of steps have been identified for the delimiters extraction: 

Step 1: Object Labeling. In this step each object from the occupancy grid is labeled 
with a unique identifier. 
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Step 2: The Contour Extraction. We compute the contours of the non-drivable 
blobs (objects, traffic isles) from the occupancy grid. Each contour point will 
represent a single cell in the grid map. 

Step 3: The Polygonal Approximation. Given a curve C we will find a polygon that 

closely approximates C while having as small a number of vertices as possible. 

Next, we will present several algorithms developed by us for delimiters extraction. 
All these methods have in common the 1 st and 3 rd step. The 2 nd step is different in 
each case. We have used two main approaches for the contour extraction: 

1) The Contour Tracing for a given object - once an object cell has been identified, 
contour tracing is performed starting from this point, adding each traversed cell to the 
current contour. In this paper we present an improved version of contour tracing, the 
3A Tracing Algorithm. 

2) The Border Scanning - a radial scanning is performed with a given radial step, 
traversing the interest zone and accumulating the contour points at the same time. The 
main difference of this approach is that we scan only the visible parts from the ego- 
car position. Two main improvements of the Border Scanning method are discussed: 
the Border Scanning using a variable step, and the Combined Border Scanning, taking 
into account the occupancy grid blob's nature (traffic isles, obstacles). 



Region of interest 




contour tracing 



c) 



Fig. 4. Contour tracing of the care points (b) from the scene (a). There are cases when two 
polygonal segments can intersect each other (c), after the polygonal approximation of the car 
contour. 
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3.1 The 3 A Tracing Algorithm 

The classical contour tracing algorithm collects the contour points of an object by 
traversing the object boundary. 

A disadvantage of the classical algorithm is that there are cases when the same 
delimiter point can be passed many times. This may lead to the incorrect 
representation, after the contour approximation step (see Fig. 4). 
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Fig. 5. 3A Tracing Algorithm. In the Accumulation phase, all traversed points are pushed onto 
the Stack A. In the Adjustment stage, the already passed points are extracted form the Stack A 
and pushed onto the Stack B. Polygonal approximation is applied in the last step of algorithm. 

To avoid this problem we have developed an extended contour tracing algorithm 
named 3 A Tracing. In this method we use two stacks, Stack A and Stack B. The name 
3A Tracing comes from the next three main phases (see Fig. 5): 

Phase 1: Accumulation. The tracing is made analogue to the Contour Tracing 
algorithm. All accumulated points are pushed onto the stack A. The traversed points 
are marked with a flag in order to know whether they were traversed or not at least 
one time. Once we found a terminal point (from which the tracing is made in the 
inverse sense) we pass to the 2nd phase of the algorithm. 

Phase 2: Adjustment. In this phase the tracing continues in the inverse sense by 
extracting already passed points (drawn with light green) from the Stack A, and 
pushing them onto the Stack B. The Adjustment is repeated until we reach a contour 
point that has not passed yet. Once the new contour point is found we pass to the 3rd 
phase of the algorithm. 

Phase 3: Approximation. Polygonal approximation is applied to each of the two 
stacks. After the polygonal approximation process the two stacks will be cleared and 
the algorithm is repeated from the Phase 1 . 

The algorithm stops when the start point is reached once again. 

Although the 3A Tracing algorithm eliminates some particular cases in which two 
polygonal segments may intersect, like in the Contour Tracing, it works only on the 
connected components. Therefore this method does not take into account the cases of 
more complex objects, when a single obstacle is represented as many disjoint patches. 
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Therefore we have elaborated an extraction method through the radial scanning of 
the Elevation Map. 

3.2 The Border Scanner Algorithm 

The Border Scanner algorithm performs a radial scanning with a given radial step. 
The scanning axis moves in the radial direction, having a fixed center at the Ego Car 
position. The scanning process is made into the limits of QJrom and Q_to angles, 
thus only the interest area are scanned, where the delimiters can be detected (see Fig. 
6). Having a radial axis with a Qrad slope, QJrom < Qrad < Q_to, we try to find the 
nearest point from the Ego Car situated on this axis. In this way, all subsequent points 
will be accumulated into a Contour List, moving the scanning axis in the radial 
direction. At each radial step we verify that a new object has been reached. If a new 
label has been found then the polygonal approximation on the Contour List points is 
performed. The list will be cleared, and the algorithm will be continued finding a new 
polygon. 




Fig. 6. Border Scanning on the Occupancy Grid Points 
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Advantages: The obtained results are more close to the real obstacle delimiters from 
the scene. The problem of the complex objects presented in the case of Contour 
Tracing algorithms is eliminated. Therefore many disjoint patches that belong to the 
same object can be enveloped by a single delimiter. 

Disadvantages: A little obstacle (noise present in the occupancy grid) can occlude a 
great part from the scene, if this obstacle is too near to the Ego-Car. The scanning is 
influenced by the presence of such false obstacles. 

The Border Scanning Algorithm using Variable Step. Having a constant radial 
step, the detected pixel density will decrease with the depth distance. The distance 
between two consecutive detected pixels is greater at the far depths. The idea is that 
some important information about the delimiters can be lost at the far distances. 

A good solution is to use a scanning method with a variable step, thus the radial 
step should be adapted with the distance. 



' /P(x1.z1) 




m±. 



4 

RfxStart.zStart) 
Fig. 7. Radial angle estimation for the next step in the Variable Step Border Scanning approach 

If we have a point P](x ly zi) of a given object and a radial axis containing the point 
Pj with a radial angle Q k r „d at the k step, then we estimate the radial angle at the k+1 
step (see Fig. 7): 



Q k 



■■ arctan(- 



-)• 



(1) 



Where: 

■ Xj, zi are the coordinates of the Pj point;; 

' Xstart> Zstan are the ego-car point coordinates. 

■ d is considered the distance between any two adjacent points. 
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However, there are situations when no object point can be reached on the current 
scanning axis. Therefore we cannot estimate the radial angle for the next step, because 
we don't know the distance of the current object point from the Ego-Car. In this case, 
like in the simple Border Scanning method, we use a fixed step, until a new object 
point will be found. 

The Combined Border Scanning. We know that the occupancy grid cells are 
classified into obstacles (cars, pedestrians etc.) or traffic isles (road-parallel patches). 
If we take into account only the first nearest point from the car, many relevant objects 
delimiters may be omitted. For example, the first obstacle from the car can be a curb. 
In this case, we are interested not only in the curb delimiters but also in the delimiters 
above the curb or behind the curb. Therefore we extended our Border Scanning 
algorithm by developing a method that takes into consideration the obstacle's nature 
making a decision based on two types of information "What have we found?" and 
"What we have to find?". The algorithm consists in two passes: one for the object 
delimiters detection, and second for the traffic isles delimiters detection. 

In the Table 1 is presented the returned result when we want to find a delimiter 
taking in account the point type we have found. 



Table 1. The Combined Border Scanning method. The result is returned, taking into 
consideration the found point type. 

Delimiter' s type we Point Type we have found Returned result 

want to find 

OBJECT OBJECT FOUND 

CURB OBJECT NOT FOUND 

CURB CURB FOUND 



4 Experimental Results 

For the experimental results we have tested a set of 15 scenarios from the urban traffic 
environment using a 2.66GHz Intel Core 2 Duo Computer with 2GB of RAM. 

Fig. 8 shows a comparative result between the Contour Tracing and 3A Tracing 
algorithms, using an approximation error of two points. One can notice that the 
polygonal segment intersection in the case of classical contour tracing algorithm (see 
Fig. 8.b) was eliminated by applying the 3A Tracing algorithm (see Fig. 8.c). 

The difference between the result of delimiters detection in the case of Simple 
border Scanner and Combined Border Scanner algorithms is presented in the Fig. 9. It 
can be observed that in the case of Combined Border Scanner (see Fig. 9.b) the side 
fence's delimiter is detected in spite of his position behind the curb (see Fig. 9.c). 
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Fig. 8. Delimiters detection through the Contour Tracing algorithm (b) and 3A Tracing 
algorithm (c). The detection is performed on the occupancy grid computed from the scene (a). 




Side fence's 
missed delimiters 



a) 



b) 



c) 



Fig. 9. Border scanning of a scene (a). The side fence's delimiters are missed in the case of the 
Simple Border Scanning (b) and have been detected in the case of the Combined Border 
Scanning algorithm (c). 
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In the Table 2 the results from the Variable Step Border Scanner and Fixed Step 
Border Scanner are computed for the same driving scene. It can be observed that the 
number of detected points is greater in the case of Variable Step Border Scanner 
algorithm, thereby 11466 points, which means 28 detected points per frame in 
comparison with 22 detected points per frame in the case of Fixed Step Border 
Scanner algorithm. 

Table 2. Fixed Step Border Scanner vs. Variable Step Border Scanner 



Fixed Step Border Scanner Variable Step Border Scanner 



Number of Frames 
Detected points 

The radial step (radians) 

Points per Frames 

Average processing time per 

frame 



406 
9058 

0.01 

22 
4 ms 



406 
11466 

variable 

28 
5 ms 



The average extraction time using the 3A Tracing algorithm is about 0.7ms per 
frame and depends on the angular resolution in the case of Border Scanner approach. 

Fig. 10a shows how the radial step size variation affects the system response time 
using the Combined Border Scanning approach. 
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Fig. 10. a) The processing time vs. the radial step size, b) The detection rate vs. the radial step size 



Fig. 10b is a diagram that shows the impact of radial step size on delimiters detection 
rate using the border scanner method. We can observe that, with a higher radial step size 
we obtain an increase in processing time while the detection rate decreases. The solution 
is a tradeoff between the system processing time and detection rate. 

Fig. 11 presents results for various traffic scenes using the Combined Border 
Scanning method. For the border scanning algorithm with a radial step of 0.01 radians 
the average processing time is about 5ms and the delimiters detection rate is 98.66%. 
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Fig. 11. Object delimiters detection through the Combined Border Scanning algorithm for 
various traffic scenes. The delimiters are projected onto the Left Image and are represented as 
grids labeled as Traffic Isles (orange) or Objects (light green). The grid height is the same as 
the enveloped object by the current delimiter. 



5 Conclusions 



In this paper we present and evaluate several methods for real-time environment 
representation through the object delimiter extraction and characterization from dense 
stereovision images. The delimiters detection is based on processing the information 
provided by a 3D classified occupancy grid obtained from the raw dense stereo 
information. The result is a more compact 2.5D model for representing the 
environment, as a set of polylines. Each polyline element inherits the type (object, 
curb), position and height properties of the associated object from the occupancy grid. 

We have developed an improved Contour Tracing method named 3A Tracing 
algorithm that eliminates the situation when two polygonal segments can intersect 
each other. 

Another approach presented in this paper is the polyline extraction through the 
radial scanning of the occupancy grid. Although the tracing approach is more 
computationally-efficient, the results provided by the Border Scanner algorithm are 
more appropriate for detecting the real obstacle delimiters from the scene. The 
algorithm is able to extract only the visible area from the ego-vehicle since the 
occluded points do not offer relevant information. Using the Border Scanner 
algorithm, our system is fast and achieves a high rate of detection: 98.66%. 

As future work we propose to extend our approaches by using temporal filtering of 
delimiter points in order to improve the polyline representation accuracy. The final 
goal of our research is to develop a system for complex environments that would 
achieve high performances with respect to accuracy, confidence and real-time 
capabilities. 
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Abstract. This paper presents a feasibility study of a vision-based autonomous 
approach and landing for an aircraft using a direct visual tracking method. 
Auto-landing systems based on the Instrument Landing System (ILS) have al- 
ready proven their importance through decades but general aviation stills without 
cost-effective solutions for such conditions. However, vision-based systems have 
shown to have the adequate characteristics for the positioning relatively to the 
landing runway. In the present paper, rather than points, lines or other features 
susceptible of extraction and matching errors, dense information is tracked in 
the sequence of captured images using an Efficient Second-Order Minimization 
(ESM) method. An optimal control design is then proposed using the homogra- 
phy matrix as visual information in two distinct approaches: position-based visual 
servoing (PBVS) and image-based visual servoing (IBVS). To illustrate the pro- 
posed concepts, simulation results under realistic atmospheric disturbances are 
presented. 

Keywords: Automatic approach and landing, Vision-based control, Linear opti- 
mal control, Dense visual tracking. 



1 Introduction 

Approach and landing are known to be the most demanding flight phases in fixed-wing 
flight operations. Due to the altitudes involved in flight and the consequent nonexist- 
ing depth perception, pilots must interpret position, attitude and distance to the runway 
using only two-dimensional cues like perspective, angular size and movement of the 
runway. At the same time, all six degrees of freedom of the aircraft must be controlled 
and coordinated in order to meet and track the correct glidepath till the touchdown. 

In poor visibility conditions and degraded visual references, landing aids must be 
considered. The Instrument Landing System (ILS) is widely used in most of the in- 
ternational airports around the world allowing pilots to establish on the approach and 
follow the ILS, in autopilot or not, until the decision height is reached. At this point, the 
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pilot must have visual contact with the runway to continue the approach and proceed to 
the flare manoeuvre or, if it is not the case, to abort. This procedure has proven its relia- 
bility through decades but landing aids systems that require specific equipment are still 
not cost-effective for most of the general airports. However, in the last years, Enhanced 
Visual Systems (EVS) based on Infrared (IR) allowed to proceed to non-precision ap- 
proaches and obstacle detection for all weather conditions. The vision-based control 
system proposed in the present paper intends then to take advantage of these emergent 
vision sensors in order to allow precision approaches and autonomous landing. 

The intention of using vision systems for autonomous landings or simply estimate 
the aircraft position and attitude (pose) is not new. Flight tests of a vision-based au- 
tonomous landing relying on feature points on the runway were already referred by 
ifll whilst O present a feasibility study on pose determination for an aircraft night 
landing based on a model of the Approach Lighting System (ALS). Many others have 
followed in using vision-based control on fixed/rotary wings aircraft, and even airships, 
in several goals since autonomous aerial refueling (|3, ||4)), stabilization with respect 
to a target (13, 10), linear structures following ([7], [8], |9]) and, obviously, automatic 
landing (|10|, [11], [12|, lfl3l . |[T4l ). In these problems, different types of visual fea- 
tures were considered including geometric model of the target, points, corners of the 
runway, binormalized Plucker coordinates, the three parallel lines of the runway (left, 
center and right sides) and the two parallel lines of the runway along with the horizon 
line and the vanishing point. Due to the standard geometry of the landing runway and 
the decoupling capabilities, the last two approaches have been preferred in problems of 
autonomous landing. 

In contrast with feature extraction methods, direct or dense methods are known by 
their accuracy because all the information in the image is used without intermediate 
processes, reducing then the sources of errors. The usual disadvantage of such method 
is the computational consuming of the sum-of-squared-differences minimization due 
to the computation of the Hessian matrix. The Efficient Second-order Minimization 
(ESM) [ 1 5 1 method does not require the computation of the Hessian matrix maintaining 
however the high convergence rate characteristic of the second-order minimizations as 
the Newton method. Robust under arbitrary illumination changes [16| and with real- 
time capability, the ESM suits all the requirements to use images from the common 
CCD/CMOS to IR sensors. 

In vision-based or visual servoing problems, a planar scene plays an important role 
since it simplifies the computation of the projective transformation between two images 
of the scene: the planar homography. The Euclidean homography, computed with the 
knowledge of the calibration matrix of the imagery sensor, is here considered as the 
visual signal to use into the control loop in two distinct schemes. The position-based 
visual servoing (PB VS) uses the recovered pose of the aircraft from the estimated pro- 
jective transformation whilst the image-based visual servoing (IBVS) uses the visual 
signal directly into the control loop by means of the interaction matrix. The controller 
gains, from standard output error LQR design with a PI structure, are common for both 
schemes whose results will be then compared with a sensor-based simulation scheme 
where precise measures are considered. 
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The present paper is organized as follows: In the Section [2j some useful notations 
in computer vision are presented, using as example the rigid-body motion equation, 
along with an introduction of the considered frames and a description of the aircraft 
dynamics and the pinhole camera models. In the same section, the two-view geometry 
is introduced as the basis for the IBVS control law. The PBVS and IBVS control laws 
are then presented in the Section[3]as well as the optimal controller design. The results 
are shown in Section|4]while the final conclusions are presented in Section[5] 



2 Theoretical Background 

2.1 Notations 

The rigid-body motion of frame b with respect to frame a can be expressed as 



a X = Q R h 6 X 



% 



(1) 



where a R(, G §0(3) and a t b G R denote the relative rotation matrix and translation 
vector, respectively, and a X G K 3 the coordinates of a 3D point in frame a. Alterna- 
tively, in homogeneous coordinates, the same relation can by given by 



a X = a T b b X = 



aTj ai 
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(2) 



where, a T b G §E(3), denotes a matrix of zeros with the appropriate dimensions and 
a X G R 4 are the homogeneous coordinates of the point a X. In the same way, the Cori- 
olis theorem applied to 3D points can also be expressed in homogenous coordinates, as 
a result of the derivative of the rigid-body motion relation in Eq. (0, 
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where, the angular velocity tensor co G R dXci is the skew- symmetric matrix of the an- 
gular velocity vector u> such that u x X = DX and the vector a V a f, = [v, ui] G R 6 
denotes the velocity screw and indicates the velocity of the frame b moving relative 
to a and viewed from the frame a. Also important in the present paper is the defini- 
tion of stacked matrix, denoted by the superscript " s " , where the matrix columns are 
rearranged into a single column vector. 



2.2 Aircraft Dynamic Model 

Let To be the earth frame, also called NED for North-East-Down, whose origin co- 
incides with the desirable touchdown point in the runway. The latter, unless explic- 
itly indicated and without loss of generality, will be considered aligned with North. 
The aircraft linear velocity v = [u,v,w] G R 3 , as well as its angular velocity 
uj = [p,q,r] G R 3 , is expressed in the aircraft body frame T c whose origin is at 
the center of gravity where u is defined towards the aircraft nose, v towards the right 
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wing and w downwards. The attitude, or orientation, of the aircraft with respect to the 
earth frame T$ is stated in terms of Euler angles <P = [<fi, 9, ip] C R 3 , the roll, pitch 
and yaw angles respectively. 

The aircraft motion in atmospheric flight is usually deduced using Newton's second 
law and considering the motion of the aircraft in the earth frame !Fq, assumed as an 
inertial frame, under the influence of forces and torques due to gravity, propulsion and 
aerodynamics. As mentioned above, both linear and angular velocities of the aircraft 
are expressed in the body frame T\> as well as for the considered forces and moments. 
As a consequence, the Coriolis theorem must be invoked and the kinematic equations 
appear naturally relating the angular velocity rate u> with the time derivative of the Euler 
angles 4> — R -1 a; and the instantaneous linear velocity v with the time derivative of 
the NED position [N, E,D] T = S T v. 

In order to simplify the controller design, it is common to linearize the non-linear 
model around a given equilibrium flight condition, usually a function of airspeed Vq 
and altitude ho. This equilibrium or trim flight is frequently chosen to be a steady wings- 
level flight, without presence of wind disturbances, also justified here since non-straight 
landing approaches are not considered in the present paper. The resultant linear model 
is than function of the perturbation in the state vector x and in the input vector u as 

(3) 

describing the dynamics of the two resultant decoupled, lateral and longitudinal, mo- 
tions. The longitudinal, or vertical, state vector is x„ = [u,w,q,9] £ K 4 and the 
respective input vector u^ = [Se, St] £ K 2 (elevator and throttle) while, in the lat- 
eral case, the state vector is Xfc, = [v,p, r, <fi] £ M. and the respective input vector 
Uft, = [6 a, tin] £ R 2 (ailerons and rudder). Because the equilibrium flight condition 
is slowly varying for manoeuvres as the landing phase, the linearized model in Eq. (f3]l 
can be considered constant along all the glidepath. 

2.3 Two-Views Geometry 

The onboard camera frame T c , rigidly attached to the aircraft, has its origin at the center 
of projection of the camera, also called pinhole. The corresponding z-axis, perpendic- 
ular to the image plane, lies on the optical axis while the x- and y- axis are defined 
towards right and down, respectively. Note that the camera frame T c is not in agree- 
ment with the one usually defined in flight mechanics. 

Let V be a 3D point whose coordinates in the current camera frame C X could be 
related with those *X in a second camera frame T*, denoted reference camera frame, by 

c X = c R**X + % (4) 

Considering that V lies on a plane 77 defined by the unit normal vector * n £ M 3 and 
the distance d* such as 

— *n T *X=l (5) 

d* 
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Fig. 1. Illustration of the considered frames and notations 



there exists a linear transformation C H» between both coordinates given by 
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where C H* G 



»:5x:5 



is the so-called Euclidean homography matrix. Therefore, given a 



set of matched point, lines or even dense information from two images X and X* , it is 
possible to compute the projective homography matrix G G M 3x3 up to a scale factor. 
Knowing an estimative of the calibration matrix K, the Euclidean homography matrix 
H can then be computed also up to a scale factor 



H = K X GK 



(7) 



The calibration matrix, or the camera intrinsical parameters, K G K 3x3 is defined as 
follows 

fx fs p XQ 
K = /„ Pvo (8) 

1 

where, the coordinates po = [p Xo ,p yo ,l] G R 3 define the principal point, corre- 
sponding to the intersection between the image plane and the optical axis, s is the skew 
factor (zero for most of the cameras) and finally, f x and f y are the focal lengths in both 
directions such that when f x = f y the camera sensor presents square pixels. 

3 Vision-Based Automatic Approach and Landing 

3.1 Visual Tracking 

The visual tracking is achieved by directly estimating the projective transformation be- 
tween the image taken from the airborne camera and a given reference image. The 
reference images are then the key to relate the motion of the aircraft Th, through its 
airborne camera T c , with respect to the earth frame Tq. For the PBVS scheme, it is 
the known pose of the reference camera with respect to the earth frame that will allow 
us to reconstruct the aircraft position with respect to the same frame. In what concerns 



274 T.F. Goncalves, J.R. Azinheira, and P. Rives 

the IBVS, where the aim is to reach a certain configuration expressed in terms of the 
considered feature, the path planning is then an implicit need of such scheme. For ex- 
ample, if lines are considered as features, the path planning is defined as a function of 
the parameters which define those lines. In the present case, the path planning shall be 
defined by images because it is the dense information that is used in order to estimate 
the projective homography C G*. 

3.2 Linear Controller 

The standard LQR optimal control technique was chosen for the controller design, 
based on the linearized models of both longitudinal and lateral motions in Eq. (0). 
Since not all the states are expected to be driven to zero but to a given reference, the 
control law is more conveniently expressed as an optimal state error feedback. The 
objective of the following vision-based control approaches is then to express the re- 
spective control laws as a function of the visual information, which is directly or in- 
directly related with the pose of the aircraft. As a consequence, the pose state vector 
P = [n, e, d, 4>, 9, ip] e M 6 , in agreementto the type of vision-based control approach, 
is given differently from the velocity screw V = [u,v,w, p, q, r] G R 6 , which could 
be provided from an existent Inertial Navigation System (INS) or from some filtering 
method based on the estimated pose. Thus, the following vision-based control laws are 
more correctly expressed as 

u=-k P (P-P*)-k v (V-V*) (9) 

where, kp and kv are the controller gains relative to the pose and velocity states, 
respectively. 

3.3 Position-Based Visual Servoing 

In the position-based, or 3D, visual servoing (PBVS) the control law is expressed in the 
Cartesian space and, as a consequence, the visual information computed in the form of 
planar homography is used to reconstruct explicitly the pose (position and attitude). The 
airborne camera will be then considered as only another sensor that provides a measure 
of the aircraft pose. 

In the same way that, knowing the relative pose between the two cameras, R and t, 
and the planar scene parameters, n and d, it is possible to compute the planar homog- 
raphy matrix H it is also possible to recover the pose from the decomposition of the 
estimated projective homography G, with the additional knowledge of the calibration 
matrix K. The decomposition of H can be performed by singular value decomposi- 
tion |T71 or, more recently, by an analytical method [18]. These methods result into 
four different solutions but only two are physically admissible. The knowledge of the 
normal vector n, which defines the planar scene 77, allows us then to choose the correct 
solution. 

Therefore, from the decomposition of the estimated Euclidean homography 

C H* =K lc G«K, (10) 
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both C R* and c t*/cf are recovered, being respectively the rotation matrix and normal- 
ized translation vector. With the knowledge of the distance d*, it is then possible to 
compute the estimated rigid-body relation of the aircraft frame T\, with respect to the 
inertial one T§ as 



c 



TV C T\ 



°R 6 °t b 
o 1 



(ID 



where, b T c corresponds to the pose of the airborne camera frame T c with respect to 
the aircraft body frame T\, and °T* to the pose of the reference camera frame JF* with 
respect to the earth frame T$. Finally, without further considerations, the estimated pose 
P obtained from °Rb and °t& could then be applied to the control law in Eq. (|9]l as 

u= -k P (P-P*)-k v (V- V*) (12) 

3.4 Image-Based Visual Servoing 

In the image-based, or 2D, visual servoing (IBVS) the control law is expressed directly 
in the image space. Then, in contrast with the previous approach, the IBVS does not 
need the explicit aircraft pose relative to the earth frame. Instead, the estimated pla- 
nar homography H is used directly into the control law as some kind of pose infor- 
mation such that reaching a certain reference configuration H* the aircraft presents the 
intended pose. This is the reason why an IBVS scheme needs implicitly for path 
planning expressed in terms of the considered features. 

Interaction Matrix. In IBVS schemes, an important definition is that of the interaction 
matrix which is responsible to relate the time derivative of the visual signal vector s G 
R fc with the camera velocity screw C V C * G R 6 as 

s = L S C V„ (13) 

where, L s G R fcx6 is the interaction matrix, or the feature jacobian. Let us consider, 
for a moment, that the visual signal vector s is a function of the Euclidean homography 
matrix C H*, the visual feature considered in the present paper. Thus, the time derivative 
of s, admitting the vector *n/d* as slowly varying, is 

s = C H„ = C R* + -5- c t**n T (14) 

a* 

Now, it is known that both C R* and c t» are related with the velocity screw C V C », which 
could be determined using Eq. (O, as follows 
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from where, C R* = c ui C R* and c t* = c v + c uj °t*. 
By using such results back in Eq. ( fPil l results on 
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Hereafter, in order to obtain the visual signal vector, the stacked version of the homog- 
raphy matrix C HJ must be considered and, as a result, the interaction matrix is given by 



c Hf 



I(3)*m/d* - C H»! 

L H 'V, = I(3)*n 2 /d* - C H» 2 (16) 

_I(3)*n 3 /d* - C H»3_ 

where, 1(3) is the 3x3 identity matrix and H^ is the ith column of the matrix as well 
as rii is the zth element of the vector. Note that, £H is the external product of ui with 
all the columns of H and u> x Hi = —Hi x u> = — Hiw. However, the velocity screw 
considered in Eq. (Qj} denotes the velocity of the reference frame T* with respect to 
the airborne camera frame T c and viewed from T c , i.e. C V C », which is not in agreement 
with the aircraft velocity screw that must be applied into the control law in Eq. ([9]). In- 
stead, the velocity screw shall be expressed with respect to the reference camera frame 
T* and viewed from aircraft body frame Tk, i.e. b V* c , where the control law is ef- 
fectively applied. In this manner, knowing the following relation from the adjoint map 



=T b b V c 
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it is possible to find the velocity screw transformation c Wt G 
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given by 
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(17) 



(18) 



such that 

C V« = - c W b b V* c (19) 

where 6 R C and b t c define the pose of the onboard camera frame T c with respect to the 
aircraft body frame Th- Using Eq. (IT6b along with the result from Eq. ( TT81 results as 
follows 

s = C H-: = L H c W b b V» c (20) 

Finally, let us consider the linearized version of the previous result as 

s - s* = c Ul - H* s = L H c W 6 b W (P - P*) (21) 

where, 



x = 



(22) 



So 
R 

is composed by the kinematic and navigation equations, respectively, linearized for the 
same trim point as for the aircraft linear model [<fi, 6, ip] = [0, 9q,0] . It is then possi- 
ble to relate the pose error P — P* of the aircraft with the Euclidean homography error 

Since the approach glidepath is a well-defined trajectory the existence of a database 
composed by images {T£}^ =0 , taken all along the reference path for equidistant lon- 
gitudinal distances dx, can be considered, as illustrated in Fig [2] The easiest way to 
defined the error function s — s* to be minimized by the control law is then to establish 



s* = H* s = I s 

such that the current and reference images match for s — s* = 0. 



(23) 
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Fig. 2. Illustration of the sequence of reference images along the path to follow along with one of 
the images from the database 



Due to the slow longitudinal dynamics of the aircraft, an acceptable result can be 
achieved using a sufficiently small distance dx between consecutive reference images 
but the size of the required database will be significant. For larger dx and consequent 
altitude errors, important oscillations on the longitudinal motion of the aircraft are ver- 
ified. An interpolation of the visual reference s* is then desirable. 

Path Interpolation. Let us consider { fc ~ 1 Hj. } , the set of Euclidean homographies, 
computed off-line, between two consecutive reference images 1^. Since the glidepath is 
essentially a straight line, it is possible to consider an interpolation in the homography 
space as 



H*( M )=I + ^( 
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such that it corresponds to an interpolation in the Euclidean space since k ^^Hk 
and, as a consequence, 
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where /i G [0, 1] is the interpolator parameter. 

The interpolator parameter /x, associated to the longitudinal distance between the 
current camera frame T c and the reference camera frame JF» , can be computed directly 
from the estimated Euclidean homography C H» as follows 



/ i=/[l,0,0]vec( c H H .- c H:) 
dx 

which, for small attitude errors and knowing that *n « [0, 1, 0] then 

c t 
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Control Law. The proposed homography-based IB VS control law is then finally given 
by 

u = -k P (L H c W ) t ( C HJ - HHm)) ~ k v (V - V*) (28) 

where, A t = (A T A) A T is the Moore-Penrose pseudo-inverse matrix. 

4 Results 

The vision-based control schemes proposed above have been developed and tested in 
an simulation framework where the non-linear aircraft model is implemented in Mat- 
lab/Simulink along with the control aspects, the image processing algorithms in C/C++ 
and the simulated image is generated by the FlightGear flight simulator. The aircraft 
model considered corresponds to a generic category B business jet aircraft with 50m/ s 
stall speed, 265m/ s maximum speed and 20m wing span. This simulation framework 
has also the capability to generate atmospheric condition effects like fog and rain as 
well as vision sensors effects like pixels spread function, noise, colorimetry, distortion 
and vibration of different types and levels. 

The chosen airport scenario was Marseille-Marignane Airport with a nominal ini- 
tial position defined by an altitude of 450m and a longitudinal distance to the runway 
of 9500m, tracking a 3° descent for an airspeed of 60m/ s. In order to present an il- 
lustrative set of results and to verify the robustness of the proposed control schemes, 
we imposed an initial lateral error of 50m, an altitude error of 30m and a steady 
wind composed by 10m/ s headwind and lm/s of crosswind. In what concerns the 
visual tracking aspects, a database of 500m equidistant images along the runway axis 
till the 100™ height, and 50m after that, was considered and the following atmo- 
spheric conditions imposed: fog and rain densities of 40% and 80%. The airborne 
camera is considered rigidly attached to the aircraft and presents the following pose 
& P C = [Am, 0, 0.1m, 0, —8°, 0] . The simulation framework operates with a 50ms, or 
20Hz, sampling rate. 

For all the following figures, the results of the three considered cases are presented 
simultaneously and identified in agreement with the legend Figure |4(a)| When available, 
the corresponding references are presented in dashed lines. 

Let us start with the longitudinal trajectory illustrated in Figure [4(a)] where it is possi- 
ble to verify immediately that the PB VS result is almost coincident with the one where 
the sensor measurements were considered ideal (Sensors). Indeed, because the same 
control law is used for these two approaches, the results differ only due to the pose 
estimation errors from the visual tracking software. For the IBVS approach, the first 
observation goes to the convergence of the aircraft trajectory with respect to the refer- 
ence descent that occurs later than for the other approaches but with a similar behavior 
with the other approaches thanks to the path interpolation. This fact is a consequence 
not only of the limited validity of the interaction matrix in Eq (l2T1 l. computed for a 
stabilized descent flight, but also of the importance of the camera orientation over the 
position, for high altitudes, when the objective is to match two images. In more detail, 
the altitude error correction in Figure [4(b)] shows then the IBVS with a slower response 
and an error after stabilization not greater than 3m as a consequence not only of the 
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Fig. 3. Screenshot of the dense visual tracking software. The delimited zone (left) corresponds to 
the bottom-right image warped to match with the top-right image. The warp transformation is the 
estimated homography matrix. 



respective estimation error but also due to the influence of the wind disturbance. In- 
deed, the path planning does not contemplate the presence of the wind, from which the 
aircraft attitude is dependent, leading to the presence of static errors. The increasing 
altitude error at the distance of 650m before the touchdown corresponds to the natural 
loss of altitude when proceeding to the pitch-up, or flare, manoeuvre (see Figure |4(c)| > 
in order to reduce the vertical speed and correctly land the aircraft. In what concerns 
the touchdown distances, both Sensors and PBVS results are again very close and at 
a distance around 500m after the expected while, for the IBVS, this distance is of ap- 
proximately 200m. It should be referred that, only a distance between reference images 
of 200m provided acceptable results for the longitudinal motion of the aircraft when 
not considering path interpolation of the IBVS scheme. The path interpolation strategy 
allowed then to reduce the size of the required data and to improve the general behavior 
of the aircraft due to the absence of discontinuities on the control actions. 

The lateral trajectory illustrated in Figure [4(e)] shows a smooth lateral error correc- 
tion for all the three control schemes where both visual control laws maintain an error 
below the 2.5m, after convergence. Once more, the oscillations around the reference are 
a consequence of pose estimation errors from visual tracking software, which become 
more important near the Earth surface due to the high displacement of the pixels in the 
image and the violation of the planar assumption of the region around the runway. The 
consequence of these effects are perceptible in the final part not only in the lateral error 



correction but also in the yaw angle of the aircraft in Figure |4(f)| For the latter, the static 
error is also an influence of the wind disturbance which imposes an error of 2.9° with re- 
spect to the runway orientation of exactly 134.8° North. As for the longitudinal motion, 
the lateral error correction for the IBVS is also here affected by the a more restricted 
regulation of the attitude angles leading to a slower and more damped response. 

We should note here the precision of the dense visual tracking software. Indeed, the 
attitude estimation errors are often below 1° for transient responses and below 0.1° in 
steady state. Depending on the quantity of information available in the near field of 
the camera, the translation estimation error could vary between the 2m and 5m for the 
lateral error, 3m and 8 for altitude and between 50m and 200m for the longitudinal 
distance. The latter is naturally less precise due to its alignment with the optical axis of 
the camera. 
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Fig. 4. Results from the vision-based control schemes (PBVS and IBVS) in comparison with the 
ideal situation of precise measurements (Sensors) 



5 Conclusions 



In the present paper, two vision-based control schemes for an automatic approach and 
landing of an aircraft using a direct visual tracking method are proposed. For the PBVS 
solution, where the vision system is nothing more than a sensor providing position and 
attitude measures, the results are naturally very similar with the ideal case. The IBVS 
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approach based on a path planning defined by a sequence of images show clearly to 
be able to correct an initial pose error and land the aircraft under windy conditions. 
Despite the inherent sensitivity of the vision tracking algorithm to the non-planarity 
of the scene and the high pixels displacement in the image for low altitudes, a shorter 
distance between the images of reference was enough to deal with potential problems. 
The absence of a filtering method, as the Kalman filter by instance, prove the robustness 
of the proposed control schemes and the reliability of the dense visual tracking. This 
clearly justifies the need of further studies to complete the validation and the eventual 
implementation of this system on a real aircraft. 
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Abstract. In this chapter being presented the information-theoretic explanation 
of Bode Sensitivity Integral - a fundamental limitation of control theory, control- 
lability grammian, Fisher Information and the issues of control under communi- 
cation constraints. As resource-economic use of information is of prime concern 
in communication-constrained control problems, we need to emphasize more on 
informational aspect which has got direct relation with uncertainties in terms of 
Shannon Entropy and Mutual Information. Bode Integral which is directly related 
to the disturbances can be correlated with the difference of entropies between 
the disturbance-input and measurable output of the system. These disturbances 
due to communication channel-induced noises and limited bandwidth are caus- 
ing the information packet-loss and delays resulting in degradation of control 
performances. 

Controllability Gramian is a measure of control performance and Fisher In- 
formation is a measure of accuracy of estimation (which needs to be done in un- 
certainties caused by packet-loss and delays of the network). Hence, in order to 
have better controllability, we need to explore the relation between the estimation 
theoretic (because information losses and delays force us to resort to estimation) 
parameter like Fisher Information Matrix (FIM) / information theoretic parame- 
ters and the control theoretic parameters of practical importance. 

Keywords: Shannon Information, Controllability Gramian, Bode Sensitivity, 
Fisher Information. 



1 Introduction 

In recent years, there has been an increased interest for the fundamental limitations in 
feedback control. Bode's sensitivity integral (Bode Integral, in short) is a well-known 
formula that quantifies some of the limitations in feedback control for linear time- 
invariant systems. In JTJ, it is shown that there is a similar formula for linear time- 
periodic systems. 

In this chapter, we are presenting the information-theoretic explanation of Bode Sensi- 
tivity Integral, a fundamental limitation of control theory, controllability gramian and the 
issues of control under communication constraints based on our paper [ 2 1 . As resource- 
economic use of information is of prime concern in communication-constrained control 
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problems, we need to emphasize more on informational aspect which has got direct re- 
lation with uncertainties in terms of Shannon Entropy and Mutual Information. Bode 
Integral which is directly related to the disturbances can be correlated with the differ- 
ence of entropies between the disturbance-input and measurable output of the system 
(refer Eq. (Q}). Entropy of the signals in the feedback loop helps to provide another in- 
terpretation of the Bode integral formula Q H as follows : 
For an LTI discrete-time system — 



H c (x)-H c (d) = ±- f \n\S(c 



]U\ 



dui 



k 



log (Pk) 



(1) 



Where S (e JW ) is the transfer function of the feedback loop from the disturbance d to 
output x and p U i 's are unstable poles (\pk\ > 1)- of the open-loop plant; S is referred 
to as the sensitivity function for an open-loop plant gain P and a stabilizing feedback 
controller gain C, S is given by S = 1+ p c - Sensitivity shows how much sensitive 
is the observable output state to input Gaussian disturbance. Here, H c (x) and H c (d) 
denote the conditional entropy of the random processes associated with the output x 
and disturbance d respectively as per FigQ] These disturbances due to communication 
channel-induced noises and limited bandwidth are causing the information packet-loss 
and delays resulting in degradation of control performances. Shannon Entropy of infor- 
mation theory is a stronger metric for uncertainty which hinders control of a system. 

It is known that control theory and information theory share a common background 
as both theories study signals and dynamical systems in general. In distributed embed- 
ded systems (DES) or networked control systems (NCS), there are issues related to both 
control and communication since communication channels with data losses, time de- 
lays and quantization errors are employed between the plants and controllers Q. To 
guarantee the overall control performance in such systems, it is important to evaluate 
the quantity of information that the channels can transfer. Thus, for the analysis of DES 
or NCS, information theoretic approaches are especially useful, and notions and results 
from this theory can be applied. The results in |6| and [7| show the limitation in the 
communication rate for the existence of controllers, encoders, and decoders to stabilize 
discrete-time linear feedback systems. 



H p | — ►QsJ-r-" 

- c \* ' 



d x 

— A s I — ► 



(a) (b) 

Fig. 1. (a)Feedback loop and (b)Sensitivity function 
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The focus of information theory is more on the signals and not on their input-output 
relation. Thus, based on information theoretic approaches, we may expect to extend 
prior results in control theory. One such result can be found in [8|, where a sensitivity 
property is analyzed and Bode's integral formula [9| is extended to a more general class 
of systems. A fundamental limitation of sensitivity functions is presented in relation to 
the unstable poles of the plants. 

2 Problem Formulation 

Networked control systems suffer from the drawbacks of packet losses, delays and 
quantization in particular. These cause degradation of control performances and un- 
der some conditions instability. Uncertainties due to packet losses, delays, quantiza- 
tion, communication channel induced noises etc. have a great influence on the system's 
performance. If we consider only the uncertainties induced by channel noise and quan- 
tization we may write: 

x(t) = Ax(t) + Bu(t) + w(t); (2) 

y{t) = Cx(t) + Du(t) + v(t); 

where A e j{ nxn [ s the system or plant matrix and B G R nxq is the control or input 
matrix. Also, x{t) is the state, u{t) is the control input, y(t) is the output, C is the out- 
put or measurement matrix, D is the Direct Feed matrix, w(i) and v(t) are the external 
disturbances and noises of Gaussian nature respectively. 

Our aim is to achieve better control performance of system by tackling these un- 
certainties using Shannon's Mutual-Information, Information-Theoretic Entropy, Fisher 
Information and Bode Sensitivity. We present the information-theoretic model of such 
uncertainties and their possible reduction using information measures. 

3 Preliminaries 

By means of the connection between Bode integral and the entropy cost function, 
paper IflOl provided a time-domain characterization of Bode integral. The traditional 
frequency domain interpretation is that, if the sensitivity of a closed-loop system is de- 
creased over a particular frequency range - typically the low frequencies the designer 
"pays" for this in increased sensitivity outside this frequency range. This interpreta- 
tion is also valid for the time-domain characterization presented in [10| provided one 
deals with time horizons rather than frequency ranges. Time-domain characterization 
of Bode's integral shows how the frequency domain trade-offs translate into the time- 
domain. Under the usual connection between the time and frequency domains: low 
(high) frequency signals are associated with long (short) time horizons. In Bode's re- 
sult, it is important to differentiate between the stable poles, which do not contribute to 
the Bode sensitivity integral and the unstable poles, which do. Time-varying systems 
which can be decomposed into stable and unstable components are said to possess an 
exponential dichotomy. What the exponential dichotomy says is that the norm of the 
projection onto the stable subspace of any orbit in the system decays exponentially as 
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t — ► oo and the norm of the projection onto the unstable subspace of any orbit decays 
exponentially as t — > — oo, and furthermore that the stable and unstable subspaces are 
conjugate. The existence of an exponential dichotomy allows us to define a stability 
preserving state space transformation (a Lyapunov transformation) that separates the 
stable and unstable parts of the system. 

3.1 Mutual Information 

Mutual Information I(X;Y), between X as the input variable and Y as the output 
variable, has the lower and upper bounds given by the following: 

R(D) = Rate Distortion = MinI(X; Y) 

C = Communication Channel Capacity = MaxI(X; Y) 

where D is the distortion which happens when information is compressed (i.e. fewer 
bits are used to represent or code more frequent or redundant informations) and en- 
tropy is the limit to this compression i.e. if one compresses the information beyond 
the entropy limit there is a high probability that the information will be distorted or 
erroneous. This is as per Shannon's Source Coding Theorem. We code more frequently 
used symbols with fewer number of bits and vice-versa. 

3.2 Shannon Entropy 

Recalling the definition of Shannon Entropy ifTTIl : 

H=-Yjnlog 2 (pi) (3) 

i 

where pi is the probability of the alternative i. The above quantity is known as the binary 
entropy in bits as we use logarithmic base of 2 (with logarithmic base e the entropy is 
in nats), and was shown by Shannon to correspond to the minimum average number of 
bits needed to encode a probabilistic source of N states distributed with probability Pi. 
The term entropy is associated with the uncertainty or randomness whereas information 
is used to reduce this uncertainty. 

Uncertainty is the main hindrance to control and if we can reduce the uncertainty by 
getting the relevant information and utilizing the information properly so as to achieve 
the desired control performance of the system. Many researchers have posed the same 
question: "How much information is required for controlling the system based on ob- 
served informations in the case where these informations are passed through commu- 
nication channels in a NCS?" 

Mutual Information I(X; Y) and Entropies H(X), H(Y) and joint entropy H(X, Y) 
are related as : 

I(X; Y) = H{X) + H{Y) - H{X, Y) 

where H(X) is the uncertainty that X has about Y, H(Y) is the uncertainty that Y has 
about X, and H(X, Y) is the uncertainty that X and Y hold in common. Information 
value degrades over time and entropy value increases over time in general. The condi- 
tional version of the chain rule 1121: 
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I(X; Y) = H(X) - H{X\Y) = H(Y) - H(Y\X) ; valid for any random variables 
X and Y. 

Mutual information I(X; Y) is the amount of uncertainty in X, minus "the amount of 
uncertainty in X which remains after Y is known", which is equivalent to "the amount 
of uncertainty in X which is removed by knowing Y". This corroborates the intuitive 
meaning of mutual information as the amount of information (that is, reduction in un- 
certainty) that each variable is having about the other. 

Just as entropy 1 13 1 in physical systems tends to increase in the course of time, the 
reverse is true for information about an information source : as information about the 
source is processed, it tends to decrease with time, becoming more corrupt or noisy 
until it is evidently destroyed unless additional information is made available. Here, in- 
formation refers to the case of desired messages. Information value degrades over time 
and entropy value increases over time in general. We can use the timely information (in- 
formation as and when it is generated) through feedback-loop to reduce the uncertainty 
and thus achieving better controllability because uncertainty in terms of information- 
theoretic entropy is the obstruction in the path of controllability. Lack of information (of 
appropriate quality and time-value) is the degradation in control performance in terms 
of controllability, stability etc. 

3.3 Bode Integral 

As pointed out and proved by researchers in Q, ifTOl . iPPfl . lfT31 that a stable system 
needs no information on its internal state or the environment to assure its stability. For 
unstable systems the mutual information between the initial state and the output of the 
system is related to its unstable poles. Stable poles do not contribute. 

The simplest (and perhaps the best known) result is that, for an open loop stable 
plant, the integral of the logarithm of the sensitivity is zero; i.e. 

In | So 0^)1 dio = 
o 

Where, So and to being the sensitivity function and frequency respectively. 

For linear systems Bode Integral is the difference in the entropy rates between the 
input and output of the systems which is an information-theoretic interpretation. For 
nonlinear system (if the open loop system is globally exponentially stable and has fad- 
ing memory) this difference is zero. Fading Memory Requirement is used to limit the 
contributions of the past values of the input on the output. Fading is the commonly en- 
countered phenomenon in wireless networks wherein the signals fade away (attenuates 
and varies with time and space). 

Entropy of the signals in the feedback loop help provide another interpretation of the 
Bode integral formula [3|, [4| as follows - Shannon Entropy - Bode Integral Relation 
can be rewritten as : 



H c (x) - H c (d) = ±- [ In | S (e? u ) | dw = £ log ( Pk ) 

Z7T J-7T . 



(4) 



Consider a random variable x E 5ft m of continuous type with entropy associated with 
this is given by 
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H ( x ) '■= ~ JsR™ ^( x ) ^P(x)dx; 

where p(x) being the probability density function of x and the conditional entropy of 

order n is defined as 

H(xk\Xk-l, ■ ■ ■ ,Xk-n) ■= - f^ m p(-)^p(-)dx 

where p(.) = (xk\Xk-i,---,Xk-n)- 

This conditional entropy is a measure of the uncertainty about the value of x at time 
k under the assumption that its n most recent values have been observed. By letting n 
going to infinity, the conditional entropy of Xk is defined as 

H c {xk) '■= lirrin—Kx) H(xk\%k-i, ■ ■ ■ i %k-n) assuming the limit exists. 

Thus the conditional entropy is a measure of the uncertainty about the value of x at 
time k under the assumption that its entire past is observed. Difference of conditional 
entropies between the output and input is nothing but the Bode sensitivity integral which 
equals the summation of logarithms of unstable poles. 

For a stationary Markov process, conditional entropy [ 12| is given by 



H(Xk\Xk-l,...,Xk-n) = H(x k \x k - 



i ■ 



It is well known that the sensitivity and complementary sensitivity functions repre- 
sent basic properties of feedback systems such as disturbance attenuation, sensor-noise 
reduction, and robustness against uncertainties in the plant model. Researchers have 
worked earlier on the issues of relating the entropy and the Bode Integral and comple- 
mentary sensitivity. For details refer to the references cited in paper (2- 

In iTTol the sensitivity integral is interpreted as an entropy integral in the time domain, 
i.e., no frequency-domain representation is used. 

3.4 Fisher Information Matrix 

When we cannot get the exact information or the information gets distorted / corrupted 
/ lost due to communication induced noises and the channel properties, we need to es- 
timate the information and hence we need the Estimation Theory. Fundamental limit of 
Estimation Theory is the Fisher Information Matrix (FEM) and the related parameter 
called Fisher Information is of paramount importance. 

Fisher information is the amount of information that an observable random variable 
Y carries about an unobservable parameter X upon which the likelihood function of X 
represented by L(X) — p(Y; X) depends. It is a measure of accuracy in estimating a 
parameter. Fisher Information Matrix is given by 

FIM X := -^( glSMfezty 

is the (Bayesian) FIM, where x is the input state, y is the observation at the output 
and E is the expectation operator . 
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Also, FIM = -E 



( d\np(f) \ f ainp(/) V 
\ dx j \ dx j 



p(f) being the Gaussian Probability Density Function of random variable /. Recalling 
the basics of FIM we can say that for Gaussian distribution it is related to the variance 
by Cramer-Rao Bound (CRB). CRB is a general uncertainty expressing reciprocity be- 
tween the mean-square-error (e) in an estimate and the Fisher Information FI present in 
the observed data. Mathematically, 

e? > -fa ot FI > $ 

It is a measure of accuracy in estimating a parameter. In other words, 
FI > 4r ; u 2 being the variance 

Lower the variance, higher the Fisher Information and vice-versa. 

Mathematically, curvature or sharpness has got a definite relation with second deriva- 
tive. To a great extent Fisher Information defines what can and what can not be known 
about a given physical scenario. FIM allows detection of some non-stationary be- 
haviour in situations where Shannon Entropy shows limited dynamics. B. Roy Frieden 
has characterized FIM as a versatile tool to describe the evolution laws of physical sys- 
tems. By information on an unknown parameter 9 contained in a random variable or in 
its distribution, we mean the extent to which uncertainty regarding the unknown value 
of 9 is reduced as a consequence of an observed value of the random variable. If there is 
a unique observation with probability 1 corresponding to each value of the parameter, 
we have a situation where the random variable has the maximum information. On the 
other hand if the random variable has the same distribution for all values of the param- 
eter, there is no basis for making statements about 9 based on an observed value of the 
random variable. The sensitiveness of a random variable w.r.t a parameter may then be 
judged by the extent to which its distribution is altered by a change in the value of the 
parameter. For details readers may refer to [IV]. 

The sensitivity of the random variable w.r.t. changes in the parameter may then be 
judged by examining the Fisher Information Matrix (FIM) as a whole. 

For Gaussian Probability Distribution, Shannon Entropy is given by 



Where, Fisher Information is FI — -\ 

=5> Fisher Information varies faster than Shannon Entropy. 

Shannon's measure of information is valid for Hamiltonian Systems IfTSl whereas 
Fisher information is valid for arbitrary dynamical systems. Fisher information is more 
general character than Shannon's or Kullback's. Fisher information is a local measure 
as it is sensitive to the local behaviour of the probability density. Fisher information 
decreases under coarse graining (i.e. less precision or quantization accuracy!) and it 
also decreases with time. 
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One has to gather relevant information, transmit the information to the relevant agent, 
process the information, if needed, and then use the information to control the system. 
The fundamental limitation in information transmission is the achievable information 
rate (i.e. a fundamental parameter of Information Theory), the fundamental limitation 
in information processing is the Cramer-Rao Bound (CRB) which deals with Fisher 
Information Matrix (FIM) in Estimation Theory, and the fundamental limitation in in- 
formation utilization is the Bode Integral (i.e. a fundamental parameter of Control The- 
ory), seemingly different and usually separately treated, are in fact three sides of the 
same entity as per 1~19). Hence, there is a need of dealing the control problem from 
information-theoretic point of view. 

Even Kalman et al. in their paper [20| have stated that Controllability Gramian 
Matrix W c - matrix is analogous to FIM and the determinant dct W c is analogous 
to Shannon Information. 

These research work motivated us to investigate some important correlations 
amongst mutual information, entropy, FIM and design control parameters (like con- 
trollability gramian) of practical importance rather than just concentrating on stability 
issues. 

4 Information Induced by Controllability Gramian 

In general, from the viewpoint of the open-loop system, when the system is unstable, the 
system amplifies the initial state at a level depending on the size of the unstable poles 
lT4ll . Hence, we can say that in systems having more unstable dynamics, the signals 
contain more information about the initial state. Using this extra information (in terms 
of mutual information between the control input and the initial state) we can reduce the 
entropy (uncertainty) and thus rendering the observation of initial state easier. 

Suppose that we have a feedback control system in which control signal is sent 
through a network with limited bandwidth. We will consider the case where the state 
of the system is measurable and the controller can send the state of the system without 
error. Under these conditions we may write: 

x(t)= Ax{t)+Bu*(t); (5) 

u*(t) = -K c x(t)+u e (t); 

where K c , u*(t) and u e (t), represent, respectively, the feedback controller gain, the 
applied control input and control error due to quantization noise of limited bandwidth 
network. In the sequel we are supposing that the control signal errors are caused by 
Gaussian White Noise which may be given by uf(t) = \/DiS(i). So we may write : 

±{t) = {A- BK c )x{t) + Bu e (t); (6) 

u*{t) = -K c x(t)+u e (t); 

or more compactly: 

x(t) = A c x{t) + Bu e (t); (7) 

u*(t) = -K c x{t)+u e (t); 
where, A c = A - BK C . 
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The feedback system (0 is a stable one which is perturbed by quantization errors or 
noises due to the bandwidth limitation. 

Lemma : The controllability gramian matrix W c of system can related with the 
information-theoretic entropy H as follows ||2T|| : 

1 n 

H(x,t) = -ln{detW c (D,i)} + -(l+ln27r) (8) 

= Average a priori uncertainty of the state x at time t for an order n of the system. 

Where D being the Diagonal Matrix (positive definite symmetric matrix) with Di being 

the jth diagonal element. Here unit impulse inputs are considered. 

and 

W c (D,r)= f e A ' t BHB T e Ajt dt 
Jo 

for a system modeled as 01. 

Proof of EqnM ■ 

Referring to IfTZl we are providing the proof. The input of (0 being Gaussian White 
Noise, the state of the system is with probability density having mean- value x(t) = 
e Act x(0) and Covariance Matrix £ at time t is given by 

S = E {{x - x)(x - x) T ) = W C (D, t). 

In a more detailed form : 

x{t) = e Ac *x(0) + I eAo'Bu'tydt 
Jo 

E{x(t)} = EL A " t x(0)+ J e A ^Bu e {t)dt 

=► x(t) = e A ^x(0) 
where x(t) denotes the mean value of x(t) and Covariance Matrix 

£ = E{{x~x)(x-xf} 



:*E = E le^'.IOl+r e Act Bu"(t)dt - e Ac *x(0)| |e Ac *x(0) + / T e Act Bu" (t)dt - c Ac 'x(0)| J 

Therefore, S = J Q r e A - t BDB T e Ajt dt = W C (D, r). 
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where, uf(t) — \fDl8(t) i.e. weighted impulses and D being the Diagonal Matrix 
(positive definite symmetric matrix) with Di being the ith diagonal element. Here, unit 
impulse inputs are considered. 



p(x,t) 



1 



(27r)"/2{detW c (D,t)} 



1/2' 



-l/2{(x-x) T V/- 1 (B,t)(x-x)}] 



(9) 



Now, for multidimensional continuous case, entropy (precisely differential entropy) of a 
continuous random variable X with probability density function f(x) (if/_ f(x)dx = 
1 ) is defined [?] as: 

Differential Entropy h(X) = — J„ f{x) In f(x)dx; 

where the set S for which f(x) > is called the support set of X. 

As in discrete case, the differential entropy depends only on the probability density 
of the random variable and therefore the differential entropy is sometimes written as 
h(f) rather than h(X). Here, we call differential entropy as H(x, t) and f(x) as p(x, t) 
which are correlated as 

tf(M) = -/rf*,«)M*.«>* m 

Using equation (O in equation ( TTOt we get 

H(x,t) = -fp(x,t) \—\{x -S) T W,T 1 (D, t)(x-x) -ln{(27r)" /2 {dot W C (D, t)} 1/2 }1 dx 



H(x,t) = \E 



E {(X, - X t )(W^ 1 (D,t)) ZJ (X J - X 3 )} 



+ iln[(2ir)"{detW c (D,t)}] 



+ iln[(27r)™ {dctW c (D, {)}] 



= hT.[E {{X 3 - X,)(X Z - X,)} (Vi-^D^t))^] +iln[(27r)"{detW c (D,t)}] 
= jEE(Wc(D, t)) 3 ,(W c - 1 (D, t)) H + I In [(2tt)" {dot W C (D, t)}] 

3 i 

= iE{(W c (D,t))(W- 1 (D,t))} 33 + iln[{(2 7 r)"{dotW c (D,t)}] 

3 

= |I?« + ^ln[{(27r) B }{detW e (D,t)}] 



(Where Ijj is the Identity Matrix) 



f + i In [{(2tt)"} {det W c (D, *)}] = f + \ In {(27r)"} + \ In {det W C (D, *)} 



f ln{(27r)} + iln{detW c (D,i)} 



:.H{x,t) = |ln{detW c (D,t)} + f(l + ln27r) 
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Based on the equation (O we can write the entropy reduction as 

AH(x,t) = |-4[ln{detW c (D,i)}] 

This shows that the entropy reduction which is same as uncertainty reduction is de- 
pendent on Controllability Gramian only. Other term being constant for constant n, 
gets canceled. 

Therefore, AH fat) = H(x(t t ),ti) - H(x(t 2 ),t 2 ) 
= iln{detW cl (D 1 ,i 1 )}-iln{detW c2 (D 2 ,t 2 )} 

For simplicity we denote AH(x, t) by AH, W c i (Di,£i) by W c i and W c2 (D 2 , t 2 ) by 
W c2 . 

Therefore, AH = \ In {det(W cl W^ 1 )} 

=> det(W c iW- 2 1 ) = e 2 ^ AH ) 

Using the above expression along with the concept of mutual information being the 
difference of the entropy and the residual conditional entropy i.e. I(X; U) = H(X) — 
H(X\U) (gain in information is reduction in entropy), we can conclude that Mutual In- 
formation I(X; U) between the state X and control input U denoted simply by Shannon 
Information I s h is given by this AH which can be expressed further as 

Finally, 

dettWeiWJa 1 ) = e 2 ( AH > = e 2/ - (12) 

We may conclude that the uncertainty reduction which is directly related to the AH(x, t) 
will reduce the variance of the state with respect to the steady-state if AH(x, t) converges 
to zero. The only influence we have on the control signal is related to that of feedback gain 
ans scheduling of control signals, to be chosen such that the norm of gramians, repre- 
sented by dct(W c (Z?i, £)) converge rapidly to their norm to infinity det(W c (Doo, oo)). 
We can also conclude from dT2l that gramian is directly related to the entropy and in 
turn with Shannon mutual information. 

5 Information Induced by FIM and Controllability Gramian 

The relation between controllability gramian, Shannon entropy and Shannon Informa- 
tion has already been established in the previous section(s). Referring to the result ( fT2t 
of previous section of this chapter we can say that 

dettWeiW^ 1 ) = e 2 ^ = e 2/ - (13) 

where W c i, W c2 denote controllability gramians and H and I s h denote the Shannon 
Entropy and Shannon Information respectively. 
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Fisher Information is also a measure of the intrinsic accuracy of a distribution and 
in lfl9ll it is shown that Mutual Information 7 s /> is related to the Fisher Information 
det(FIM) as follows : 

7 s/l = ±ln{det(FIM)} 

Therefore, 

det(FIM) = e 2hh (14) 

Hence, using equations (fT4l and (fl~3T > we can conclude that 

det(FIM) = det(W c iW t 7 2 1 ) (15) 

Thus with the reduction of controllability gramian norm from W_cl to W_c2 there is an 
increase in FIM and higher the reduction of controllability gramian, more is the Fisher 
information associated with. Since Cramer-Rao Bound implies that Fisher information 
is the reciprocal of the variance as its lower limit and controllability gramian is same 
as variance for a white Gaussian noise as input, we can conclude that our result is in 
agreement with the already established theories. 

From control theory the control energy expression is given by 

e 2 .c = x T W _1 _CX 

By controllability one means that with minimum energy the state of a system can be 
steered towards the target in a finite time. Minimizing the energy means minimizing 
W _c i.e. maximizing W_c in the sense of a given matrix norm. The use of enriched 
Fisher information which has got a direct relation with variance and controllability 
gramian helps us to achieve the target state with greater ease and prudent transfer of 
information which is of great interest in communication and power constrained embed- 
ded systems. 

An important observation in the above energy expression (control-theoretic) is that 
the locus is an ellipsoid with axis direction given by eigenvector of \V_c~ 1 and length 
proportional to eigenvalue. Smaller the ellipsoid is, more controllable the system is. 
Similar analogy can be drawn in information theory lfl3l as well : 

AI(Y;X) = -(AX) T FIM{AX) 

where superscript T denoting Transpose, /, X and Y being the mutual information, 
input and output processes respectively with A being the change in respective 
parameters. 

It is shown in ll22l that the derivative of the mutual information (nats) w.r.t the signal- 
to-noise ratio (snr) is equal to half the Minimum Mean Square Error (MMSE), regard- 
less of the input statistics i.e. 

(snr) = -MMSE(snr) (16) 

dsnr y ' 2 y J 

This relationship holds for both scalar and vector signals, as well as for discrete-time 
and continuous-time non-causal MMSE estimation. MMSE reduces with snr and Shan- 
non Information increases with snr for Gaussian and Binary Channel as well. Hence, by 
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reducing quantization noises (in terms of Quantization Precision /Accuracy) in [23] at 
the time when the system approaches steady-state we can increase the snr and with in- 
crease in snr we can have increased Mutual information and thus less uncertainty. Here, 
we provide the theoretical justification of the simulation work done before in (23). 

6 Conclusions 

This chapter has addressed some new ideas based on our paper [2] concerning the 
relation between control design and information theory. Since the distributed embed- 
ded system or networked control system has communication constraints due to limited 
bandwidth or quantization noises, we must have to adopt a policy of resource allocation 
which enhances the information transmitted. This may be done possible if we know 
the characteristics of the networks, the bandwidth constraints and that of the dynamical 
system under study. 

As demonstrated here, the controllability gramian constitutes a metric of information 
theoretic entropy w.r.t the noises induced by quantization. Reduction of these noises is 
equivalent to the design methods proposing a reduction of the controllability gramian 
norm. 

Since zoom - in (near the target) is equivalent to reducing the entropy (contracting 
the uncertainty domain) and zoom - out (away from the target) is equivalent to increas- 
ing the entropy (expanding the uncertainty domain), work in this direction to propose 
an information-theoretic analysis of the zooming algorithm proposed in [23| would be 
dealt in future. 

Controllability gramian is, in some sense, a measure of energy. Higher the control- 
lability gramian (W c ), lower is the energy associated with it and hence lower the un- 
certainty (randomness, like the Brownian motion or the physical notion of energy and 
entropy). Hence, entropy reduction is associated with controllability gramian reduction 
and entropy reduction is at the cost of gain in Shannon mutual information which is es- 
tablished in this chapter vide equation ( fT2l . This chapter has addressed some new ideas 
(based on our work [24]) concerning the relation between Control Theoretic parame- 
ter (Controllability Gramian) and FIM of Estimation Theory which, in turn, linked to 
Information Theory. Since the DES or Networked Embedded Control System (NECS) 
has communication constraints due to limited bandwidth or noises, we must have to 
adopt a policy of resource allocation [ 25 1 which enhances the information transmitted. 

It is demonstrated that the controllability gramian norm constitutes a metric of 
information-theoretic entropy w.r.t the noises induced by quantization. In fact, phys- 
ical interpretation of controllability Gramian is : "If the input to the system is white 
Gaussian noise, then controllability gramian is the covariance of the state". Reduction 
of these noises is equivalent to the design methods proposing a reduction of the con- 
trollability gramian norm which is related to the Fisher information matrix as well. 

7 Future Work 

Future work in this direction would be simulation and / or experimental verification of 
the theoretical approaches we attempted to project in this chapter. 
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VAR Based State-space Structures: Realization, 
Statistics and Spectral Analysis 
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Abstract. The effective representation of multivariate time-series in terms of 
their structural indices is faced in this study, leading to the development of a cor- 
responding scheme that is based in the interconnection between VAR and state- 
space models. To this, a specific state equation that retains a state matrix identical 
to the companion matrix of the original VAR polynomial is realized, uncovering 
many "hidden" information of the initial process. In light of this realization, it is 
shown how closed form expressions for the Green function, the covariance gen- 
erating function and the spectral density can be derived through the spectrum of 
the state matrix, thus allowing assessment and quantification (via the notion of 
dispersion analysis) of every structural mode. A Yule-Walker based estimate is 
also provided, which applies directly to the state equation. A structural system 
with two degrees of freedom and closely spaced modes serves as an application 
of the novel scheme, using Monte Carlo analysis. 

Keywords: Time-series, Vector autoregressive, State-space, Green function, Co- 
variance function, Dispersion analysis, Spectral density, Estimation. 



1 Introduction 

The analysis of vector time-series, generally referring to the determination of the dy- 
namics that govern the performance of a system under unobservable excitations/distur- 
bances, has been a subject of constant development for many years, as part of the 
broader system identification framework. Relative applications are extended from 
econometrics [3 151 . to dynamics [151111 , vibration and modal analysis II18I10L con- 
trol lr9lT2ll and fault diagnosis ffl. 

The study of vector time-series can be assessed from a variety of viewpoints, with 
respect to the application of interest. These include simulation, prediction and extraction 
of structural information. Yet, while in the first two areas the interrelation of the corre- 
sponding time-series structures, such as the VAR one (or the VARX and the VARMAX, 
under the availability of input information), to equivalent state-space models has been 
studied extensively 17121 161 . not much have been done in the third 11315 1, from where 
it appears that state-space realizations may provide significant advantages, regarding 
structural estimation, with respect to other approaches [8|. 

This work attempts to provide a unified framework for the structural representation 
of vector time-series, by means of VAR models and their corresponding state-space re- 
alizations. Based on the fact that every VAR structure of order n (notated from now on 
as VAR(n) structure) can be expressed as an equivalent (and non-unique) VAR(l) one, 

J. A. Cetto et al. (Eds.): Informatics in Control Automation and Robotics, LNEE 85, pp. 301 ^-315. | 
springerlink.com © Springer- Verlag Berlin Heidelberg 201 1 
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a corresponding state-space model is developed. This specific model qualifies, over 
other possible realizations, for having a state matrix that coincides with the VAR(n) 
matrix polynomial. It turns out that the spectrum of this matrix (not to be confused 
with the spectral density of a time-series) has all the structural information about the 
system that generates the time-series "hidden" in its spectrum. Consequently, by tak- 
ing advantage of standard results of matrix algebra, closed form expressions for the 
Green function, the covariance generating function and the spectral density of the state 
and output vectors are derived, which reveal the dependence to the structural modes. 
In addition, the impact of every such mode in the total energy of the state and output 
equations is assessed, by following a methodology that has been recorded in the liter- 
ature as dispersion analysis lfT4l . Yet, unlike other dispersion analysis estimators |6| 
that rely on the calculation of the residues of an estimated transfer function, the one 
addressed here is also based on the spectrum of the state matrix and includes the effect 
of the stochastic excitation's covariance. The applied framework is independent to the 
estimation method selected, however, a state matrix estimate that is based on the Yule- 
Walker equations (resulting thus in a minimum phase (stable) estimator) is provided, 
the effectiveness of which is investigated through a simple structural system, with two 
degrees of freedom, that is characterized by a pair of closely spaced modes. 

The rest of the text is organized as follows: in Sec. [2] the VAR model is outlined 
and the reduction to state-space is introduced. Section [3] investigates the properties 
of the VAR(n) state-space realization, regarding the Green function, the covariance 
generating function and the spectral density matrix, including dispersion analysis, and 
formulates the relations with the original VAR(rt) model. Section [4] deals with the es- 
timation of the state matrix. The Monte Carlo analysis of the structural system takes 
place in section [5] and the main characteristics of the introduced method are concluded 
in section [6] 

2 The VAR State-space Realization 
2.1 Preliminaries 

r i T 

Let Y[t] = yi[t] y2[t] ... y s [t] denote a s-dimensional vector time-series of zero 

mean random variable^. Under the stationarity assumption (T), Y[t] can be described 
by a finite order VAR(n) model of the following form: 

Y[t] + A r Y[t - 1] + ... + A„-Y[f -n} = Z[t] (1) 

In the above equation n is the order of the VAR process, Aj designate the [s x s] AR 
matrices and Z[i] describes a vector white noise process with zero mean and covariance 
function, 



r z [h]^E{z[t + h]-Z T [t}} = r Z ^ (2) 

where Sz is a non-singular (and generally non-diagonal) matrix. 

1 Throughout the text, quantities in the brackets shall notate discrete-time units (or time lags, in 
the case of covariance functions) and hats shall notate estimators / estimates. E{-} shall notate 
expectation. 
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Taking advantage of the backshift operator q, defined such that q k ■ Y [t] = Y [t — k] , 
the VAR(n) structure can be compactly written as, 



A(q)-Y[t] = Z[t] 
where A(q) is the [s x s] AR matrix polynomial: 

A(q) =I S + Avq- 1 + ... + A n -q-" 



(3) 



(4) 



2.2 Reduction to State-space 

Any VAR(n) process of Eq. Q] can be transformed to an equivalent VAR(l) struc- 
ture [16|. By defining the [n-s x 1] vectors, 






'Y[t -n + 1] 

Y[t - 1] 
Y[t] 



and the \n-s x n-s] and \s x n-s] matrices, 






N[t} = 



o s o s 

~ A-n "-n— 1 



o 

my 



o s 
o s 

o s 

-Ai 



(5) 



(6) 



C = [O s O s . I s ] 
respectively, Eq.[T|can take the following form: 

S[t] = F-B[t - 1] + N[t] 



(7) 



(8) 



Y[t] = C-S[t] 



(9) 



Equations [8]-[9] illustrate the state-space realization of the VAR(n) structure of Eq. Q] 
Naturally, the state-space model consists of a state equation (Eq. [8J, in which F is 
the state matrix, and an observation equation (Eq. [9]) that relates the original s-variate 
time-series Y[i] to the state vector, S[t], by means of the output matrix C. Obviously, 
the state equation can be viewed as a VAR(l) model, in which S[t] is a well-defined 
stationary stochastic process and N[i] has properties similar to that of Z[i], as it will 
become clear at the following. 

It must be noted that the state-space realization of Eq.Q]is not unique |[T3l . In fact 
they exist infinitely many pairs {F, C} that can describe Y[t] in terms of Eqs. [8]-[9j 
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since any transformation of the state vector by a non-singular [n-sx n-s] matrix T leads 
to new state equation, in which the state matrix T-F-T -1 is similar to F and preserves 
its eigenvalues ITTIl . Yet, Eq. [8] has a very important property: the state matrix F, as 
defined in Eq. [6] is the block companion matrix of the AR polynomial described by 
Eq.|4]and includes all the structural information of interest, regarding the process that 
generates the s-variate time-series Y[t\. 

3 State-space Realization Properties 

3.1 Noise 

From Eqs.[5][7j it holds that, 

N[t] = C T -Z[t] (10) 

thus, N[i] is characterized by zero mean and covariance matrix given by, 

r N [h] = £JN[£ + /i]-N T [£]| = E\c T -Z[t + h]-Z T [t]-C\ 

= c T -.e{z[i + h]-z T [t}\-c = c T -r z [h]-c (ii) 



leading to, 



where 27 N = C T -i;z-C. 



3.2 State Vector 



r^[h}= ^ N h = (12) 

L J |0 h^O 



The Green Function. In view of Eq. [5J the state equation can be written as an infinite 
VMA, 

3[t] =^F fc -AT[t-fc] (13) 

fc=0 

which is a multivariate generalization of Wold's theorem [1|. The Green function is 
defined as: 

OO OO 

H s (g)=]TH s [fc].<r fc = ^F fc .<r fe (14) 

k=0 k=0 

Under the assumption that F retains a complete set of eigenvalues |Ai, A2, ••-, A„. s }, it 
can be expressed as, 



n- s 



3=1 
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where Gj are the, so called, spectral projectors of F (refer to Meyer [17], Chapter 7, 
for a rigorous analysis on the spectral properties of square matrices). The substitution 
of Eq.HUto Eq.Hl using the fact that G) = Gj and GjGj = 0, i^j, yields, 

oc n-s , n-s 1 

H «(g) = E[E G H'- g ~ fc = £ i-A-- g -i ' G ' < 16 > 

fc=o j=i J i=i '■> q 

so that the Green function is decomposed into n ■ s first order terms. In addition, from 
Eq. [14] the coefficients (weights) of the Green function can be expressed in a closed 
form as, 



n ■ .s 



H s [fc]=F fc = ^G,-A* (17) 

i=i 

in terms of the spectrum of F. In general, Hs [k] has a decaying performance, charac- 
terized by a mixture of damped exponentials and cosines, as for example in vibrating 
systems, where the eigenvalues Xj often appear in complex conjugate pairs. Further- 
more it holds that iflTl : 

n-s 

Hs[0]=^Gj=I (18) 

.7=1 

The Covariance Generating Function. The covariance generating function is defined 

as. 

oo 
r s (q)= J2 r S [k]-q- k (19) 

k— — oo 

where Pg [k] is the covariance matrix related to the Wold decomposition of the state 
equation (as expressed in VMA form, Eq.[T3t. given by: 



i=0 



oo oo 



rs[ft]=5^H s [* + fc]-27 N -H£[i] (20) 

Substitution in Eq.[T9l implies: 

k— — oo i— 

Setting i + k = j and assuming causality, so that Hs[fe] = O s for k < 0, Eq. [2T1 
becomes, 

oo oo oo oo 

r s (q)= Yl EH-W^N-HSW-g 4 -^ £ H s [j]-i: N -^H|W • j-i 

j — — oc i—0 j — — oc i— 

oc oo 

= ^H 3 [j]-^ ■ 27 N -£]H£[i] • <f = U s (q) ■ Zn-nKq- 1 ) (22) 

j=0 i=0 
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or, using the result of Eq.[T6l 

n-s 1 n-s 1 

^^EiT^-i-Gr^-Er^-Gj (23) 

3=1 3 H 3=1 3 H 

so that the covariance generating function of the state equation is also expressed in 
terms of the state and the noise covariance matrices. 

Of particular importance is the covariance matrix of the state vector, Ts [k] . Com- 
bining Eqs.[T7landl20l 

OO 

r B [h] = J2 Fk+h - s "-[ Fk ] T ( 24 > 

fc=0 

that leads to (the proof is given in 0), 

n-s 

r 3 [h]=Y, D r^ (25) 

3=1 

where, 



/; '.s 



D, = G r 2 N - J2 i x "\ (26) 

m— 1 J 

Equationl25lhas some important features. Firstly, as becomes directly evident, it has the 
same form as the weights of the Green function. Secondly, it describes the covariance 
matrix in terms of the spectral properties of the state matrix (plus the noise covariance), 
which, as already mentioned, contains all the information about the dynamics that pro- 
duce the state vector and, thus, ~Y[t]. This fact leads to a third crucial feature: for h = 0, 
Eq.l25lvields: 

r s [0]=D 1 +D 2 + -.- + D n . s (27) 

Recalling that Ta [0] can be treated as the multivariate equivalent of the variance (in 
fact its diagonal elements are the variances of each entry of H[i]), Eq. [27] can be used 
as a direct measure of the significance that every eigenvalue has, in the total energy 
of the vector time-series. This leads to the notion of dispersion analysis, originated in 
the work of Lee and Fassois |[T4l . in which the estimated modal characteristics of a 
vibrating system are qualified against some predefined thresholds, under a broadband 
stochastic excitation environment. Since the eigenvalues of the state matrix may be both 
real (say k\) and complex conjugate pairs (say 2 • ki, so that k\ + 2 • fc 2 —n-s), the 
same applies to the matrices Dj (see Eq.[2"6l>. Thus, by following a similar procedure to 
that described in Fassois [6], Eq.|27]can be written as, 

r s [o]=X> fc + E D * + D fe 

fc=l k=k 1 + l 

fei fei+fc 2 

k=i k=fci+i 
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where M£ and M£ denote the real and complex dispersions, respectively. Correspond- 
ingly, within the content of the {i,j} entry of the zero lag covariance matrix, the per- 
centage dispersion matrices are defined as, 

A* k = [5* ki .], *:r,c (29) 

where, 

K i} = 100 • j^% (30) 

while any matrix norm can be utilized, in order to assess the significance of a specific 
mode in the total vector time-series energy. 



The Spectral Density. For time-series sampled at a frequency F s , the spectral density 
matrix of the state vector is given by the Fourier transform of the covariance matrix, 



S 3 (/)= J2 r s [k]-e- j - 2 -^ l - k (31) 



where / is the frequency in Hz. It is worth noting that the above expression for the 
spectral density matrix retains a strong relation to the one of the covariance generating 
function. In fact, using Eq.Q~9j 

S 3 (f)=r s (q = e j -^) (32) 

thus, by substituting the resultant formula for rs(q), as described by Eq. |23j the fol- 
lowing relation occurs: 

n-8 -. n-s ^ 

Sh(/) = E— -^T-G^n-E-- -jtt-GJ (33) 

]= i 1 - Aj ■ • e J F = j =1 1 — Xj ■ e J F * 

A more efficient expression for the spectral density matrix can be derived directly from 
Eq.[5U by replacing the covariance matrix via Eq.[25l 

— 1 oo 

S s (/) = -Tb[0]+ J2 rs[k]-e- j -^" k + ^r s [k]-e-^^" k (34) 

k— — oo 

Since i~s[— k] = r^[k], EQ.l34lbecomes, 

oo 

Sb(/) = r s [0] + J2 r Si k ] ■ e r± $r- k + J^ f sW • e^'^'* (35) 

Using Eq. |25l 

s B {f) = r a [o] + '£Y, T >™< 

k—1 m=l 

oo n-s 

^D m <-e-^'' (36) 



fc— — oo fc— 1 

■<T\ 



2-x-/ 
Fs 

k=l fe=l 



oo n-s 

2-ff 



e r 



k—1 m—1 



n-s oo 



v fe a -k 
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■ 2-7,-f 

Setting a — e r F = for ease in manipulation, it follows that: 

n-s oo n-s 

s s (/) = r s [o] + J2v T m -J2^-a k + Y: d£.£a* 

m—1 k—1 m—1 k—1 

n-s oo 

= T S [0] + ]T D ™-(A™ • a) ■ £(A m • a) k 

m—1 k—0 

n-s oo 

+ /2 D "i'(A™ ■ a_1 ) • z2^ m ■ a ~ 1 ) fe 

m=l k=0 

n-s , , _1 

= r g [0] + ^D^-^-+D m . - Xm x a - x (37) 

, 1 - A m • a 1 - A m • a i 

By replacing a, 

n-s . _j. 3 -".f . j- 3 """* 

Ss(/) = r s [0] + £ A? " ,e .^ • D m + A '"' e ^ • Df„ (38) 

t 1 — \ . p 3 F s 1 — \ . p3 F s 

with D m calculated by Eq.[26| Since the spectral density matrix is Hermitian, Ss(— /) = 
S s (/) - it is sufficient to calculate and illustrate its auto and cross spectral density entries 
in the [0 F s /2] frequency interval, according to: 

3.3 Output Time-Series 

The preceding analysis explored the advantages of the state equation and led to closed 
form representations for the Green function, the covariance generating function and 
the spectral density matrix, which are exclusively depend on the spectrum of the state 
matrix F and, where applicable, the state noise covariance S^. Naturally, there exist 
strong connections between these quantities and the corresponding ones of the s-variate 
time-series Y[£], via the output equation of the state-space model. These are explored 
in the following. 

Green Function. The substitution of Eq.[l3]to Eq.[9l in view of Eq.[l2l produces, 

oo 

Y[t] = C-S[t] =C-J2 B -s[k]-N[t-k] 
k=i 

oo 

= C-J2Ks[k]-q- k -N[t] 

fe=i 
= C-H s (q)-C T -Z[t] =H v (q)-Z[t] (40) 
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thus, the Green function of original VAR(n) model is, 

oo n • s 1 

H Y (g) = J2 Hy[A] • q- k -Z[t] = ]T _ 1 C G, C T (41) 

fc=i j=i ■J ' 9 

while its weights are calculated by: 

H Y [fc] = C-H s [fc]-C T 

n-s 

= ^C-G r C T -A* 

3=1 

3=1 

Covariance Generating Function. By definition, 

oo 
r Y (q)= Y, r Y [k}-q- k (43) 

k— — oo 

The covariance matrix of Y[t] is: 

r Y [ft] = E\ Y[t + ft] • Y T [t] [ = E H Y [fc + ft] • S z • H Y [k] (44) 

*■ J fc=0 

Recalling that, 

Sit = C T I?z C (45) 

and 

H Y [fc] = C-H s [fc]-C T (46) 

the following apply: 

oo 

r-y[h] = Y c ■ H s[ fc + } A- cT ■ 2 Z -C- H|[fc]-C T 

fc=U 

= C-I ^H s [fe + ft]-^N-H|[fc] I -C T 

= C-r s [ft]-C T (47) 

Putting this result in Eq.l43l implies. 

r Y («) - C-r s (g) • C T (48) 
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thus the covariance generating function and the covariance matrix of the original VAR(n) 
model take the form 

IYG/) = C-Hs(g) • ^n-HK?" 1 ) • C T 

= C-H s (g) • C T -27z-C-Hs(g _1 ) • C T 

= H Y (g) • ^■H^f 1 ) (49) 



r Y [h]=J2Qr^ (50) 

3=1 



Qj = C-Dj ■ C T (51) 



and, 



respectively, where: 



while the dispersion analysis of the state vector (Eqs. [27l430t can be straightforwardly 
applied to Eq.[50]as well. 

Spectral Density, the spectral density matrix of the s-variate time-series Y[i] is given 

by, 

Sy(/) 



-l-k 



= C-S s (/)-C T (52) 

so that, in light of Eqs.[38]and[5T] the following expression is derived: 



E 


r Y [k]-e 


-3-*&"k 


k— — oc 






oo 






k— — oc 


c-r B [k\ 


■ C T ■ e~ j 



II ■ s 



S Y (/) = T Y [0] + ]T Am ' e " ' "L., ■ Q m + Am ' e ' F L. f ■ Cl T m (53) 

Again, since Sy(/) is Hermitian, the calculation of the spectral density can be made 
for the frequency range between the zero and the Nyquist frequencies: 

\S Y (/) ,/ = 

Equations |42] |50] and [54] show how three fundamental characteristics of the s-variate 
time-series Y[t] are related to the state matrix, F. In addition, they retain a form that 
is easy to implement and allows direct quantification of every structural mode. It is 
important to note that the above analysis is strictly depended to the spectrum of F and 
the noise covariance. Indeed, when a VAR(n) structure described by Eq.[T]is available, 
all the information about the dynamics of the system that produces Y[t] can be assessed 
by the eigenvalue problem of F, which is identical to the block companion matrix of 
the AR polynomial, A(q). 
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4 Estimation 



The estimation of VAR(n) structures pertains to the identification of the matrix poly- 
nomial order and coefficients, as well as the covariance matrix of the vector noise se- 
quence, given observations of a s-variate times-series Y[i], t — 1, ..., N, that has been 
sampled at a period T s . To this, the state-space realization may again be utilized, noting 
that, regardless the selected order n of the original VAR structure, the state equation re- 
tains the first order VAR form. Post-multiplying Eq.[8]by S T [t + h], h > 0, and taking 
expectations on both sides leads to the Yule-Walker equations: 



r B [h] =Fr s [h-i] 

The canonical form of the the state matrix allows its decomposition as, 

F = T C T f 



(55) 



(56) 



where, 



01 
00 1 



00 ... 1 
00 ...0 



1 A s I — |_-A. n A-n- 



At] 



(57) 



and C is given by Eq. [7] Thus, by selecting h — 0, the following estimate can be 
derived, 



f = C- T-f s [l] •f^ 1 [0] 



(58) 



where i~s[0] and -T|~[l] are the zero and first lag sample covariance matrices of B[t], 
respectively. Correspondingly, the state noise covariance matrix is estimated as ^n = 
f s [0]-F-f|[l]. 

It can be shown that the estimator of Eq. [58] is of minimum phase, sharing similar 
convergence properties to the least-squares one (the reader is referred to [5] and lfT6l . 
Chapter 3, for details about convergence). Having the state equation estimated, the tran- 
sition to the original VAR(n) structure is designated by the matrix C of the state-space 
realization's output equation. To this, the transformation methods that were implied in 
Sec. [3] can be applied. Notice that in fitting VAR models to data, it is interest to keep in 
mind the procedure of fitting higher-order VAR models and testing the significance of 
the last coefficient matrix, as a direct extension of the partial autocorrelation function's 
use. It is easy to observe that this procedure can be also reflected back to the eigenvalue 
problem of the state matrix: when the last coefficient approaches zero, F approaches 
singularity and some of its eigenvalues produce low norm percentage dispersions. Re- 
garding the latter, their significance can be quantified using the following steps: 

Step 1: under the availability of an estimated VAR(ri) model, apply a state-space real- 
ization (Eqs. [S]®, construct the state matrix F and calculate its eigenvalues. Say that 
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Fig. 1. A structural system with two degrees of freedom: mi = ni2 = 4.5 kg, c\ = 45 Ns/m, 
c 2 = 35 Ns/m, c 3 = 15 Ns/m, ki = k 3 = 17500 N/m, k 2 = 100 N/m 



these are split into k\ real and kz complex conjugate pairs (k\ 
dimension of the vector time-series). 



2 • &2 = n ■ s, s the 



Step 2: calculate the coefficients Qj of the covariance matrix r-y[h] (Eqs. 
spectively). 



Step 3: extend the procedure described by Eqs. 12714301 to the Q/s and calculate the, 
real and complex, percentage dispersion matrices A.^. Use any matrix norm to quantify 
the significance of the k th mode (the maximum absolute element of A.^ is used in the 
following). 



Table 1. Theoretical / identified natural frequencies (Hz) and damping ratios, and dispersions of 
the identified VAR(2) models (estimated values appear as mean( standard deviation) of the Monte 
Carlo experiments) 

Dispersion matrices (%) 



Theoretical Identified 


1 st Mode 2 nd Mode 


w n 9.98 9.99(0.13) 

9.93 9.93(0.05) 

C 0.18 0.18(0.02) 

0.05 0.05(0.00) 


' 37.50(5.40) -32.58(8.83)" 
-36.31(8.71) 9.53(2.50) 


" 62.50(5.40) 132.58(8.83)" 
136.31(8.71) 90.77(2.50) 



5 Application Study 

The method's performance was examined through the structural identification problem 
of a vibrating system with two degrees of freedom, presented in Fig.Q] The system is 
characterized by a pair of closely spaced modes, as indicated in Tab. Q] and the vec- 
tor time-series used for the identification tasks was the vibration displacement of the 
masses. The statistical consistency of the method was investigated via Monte Carlo 
analysis that consisted of 20 data records of vibration displacement time-series, with 
each such record having 1000 samples, obtained with different white excitations and 
noise-corrupted at 5% noise to signal (N/S) ratio. Regarding the simulation, the contin- 
uous system was discretized using the impulse-invariant transformation, at a sampling 
period T s = 0.025 s. 
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o 

I - 20 

^ -40 
CO 



■ Total 

1 st Mode 
- 2nd Mode 





Frequency (Hz) 



10" 

Frequency (Hz) 



T3 




-20 
-40 
-60 
-80 
-100 
-120 



Total 

■ - - 1 st Mode 
- - - 2nd Mode 




10' 

Frequency (Hz) 



10 


CD 

2. - 10 

S -20 

CM 
CM 

CO -30 

-40 
-50 



— Total 
-1st Mode 
- 2nd Mode 

10° 
Frequency (Hz) 
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Following the estimation procedure described in Sec.|4]and utilizing statistical order 
selection via the BIC criterion [5|, VAR(2) models were found adequate to describe 
system dynamics. Table Q] illustrates the estimates of the natural frequencies and the 
damping ratios, together with the theoretical ones, from where it is clear that the method 
performed satisfactory and identified the relative quantities, even in the absence of the 
input excitations. Table Q] further displays the percentage dispersion matrices for each 
mode of vibration, showing that the second mode is a heavier contributor in the total 
energy of the system. 

For further validation of the results, Figs. [2H2] display the Green function (impulse 
response) and the spectral density matrix, respectively, for one VAR(2) model out of the 
estimated set, together with the contributions from the two modes of interest. Clearly, 
the second mode dominates in both time and frequency domains, apart from the 
Hn [t]/S\i(f) impulse/frequency responses, where the first mode determines the per- 
formance. 

6 Conclusions 

A novel method for the representation of vector time-series, by means of VAR(n) struc- 
tures, was presented in this paper. Focusing on the estimation of structural information, 
the method takes advantage of the fact that every VAR(n) structure can be turn into a 
VAR(l) counterpart and is led to a state-space realization, whose state matrix coincides 
with the block companion matrix of the VAR polynomial. Consequently, it was shown 
how important quantities of the original VAR(n) structure, such as the Green function, 
the covariance generating function and the spectral density matrix, can be qualified and 
assessed only in terms of the spectrum of the transition matrix (and the noise covariance, 
where applicable). This fact provides the user with the ability to accurately evaluate the 
significance of every structural mode in the total vector time-series energy (a technique 
referred to as dispersion analysis). Of the advantages of the method is its ability to use 
the state equation for identification purposes, by implementing a Yule-Walker based 
estimate that results in (unique, for a given data set) minimum phase estimators. 

The encouraging results (reduced data acquisition, statistical consistency, accurate 
structural identification, no overdetermination, unique estimate) suggest the further re- 
search into this field. Extension of the method to vector time-series with structural in- 
dices governed by multiple eigenvalues, probably by means of Jordan canonical forms, 
as well as the investigation of VARMA models, ensues straightly. Of main interest is 
also the application of the method under the availability of input excitation and the 
expansion of its framework to non-stationary vector time-series, to closed-loop opera- 
tions, as well as to fault diagnosis schemes. 

References 

1. Box, G.E.P., Jenkins, G.M., Reinsel, G.C.: Time Series Analysis, Forecasting and Control, 
4th edn. Prentice-Hall, Inc., New Jersey (2008) 

2. Brockwell, P.J., Davis, R.A.: Introduction to Time Series and Forecasting, 2nd edn. Springer, 
New York (2002) 



VAR Based State-space Structures 315 

3. Clements, M.P., Henry, D.F.: Forecasting Economic Time Series. Cambridge University 
Press, Cambridge (1998) 

4. Dertimanis, V.K.: Fault Modeling and Identification in Mechanical Systems. Ph.D. thesis, 
School of Mechanical Engineering, Athens, Greece (2006) (in greek) 

5. Dertimanis, V.K., Koulocheris, D.V.: On the state-space realization of vector autoregressive 
structures: an assessment. In: Proceedings of the 6th ICINCO Conference. INSTICC, Milan, 
Italy (2009) 

6. Fassois, S.D.: MIMO LMS-ARMAX identification of vibrating structures-Part I: The 
method. Mechanical Systems and Signal Processing 15(4), 737-758 (2001) 

7. Hannan, E.J.: The identification and parameterization of ARMAX and state space forms. 
Econometrica 44(4), 713-723 (1976) 

8. He, J., Fu, Z.F: Modal Analysis. Butterworth-Heinemann, Oxford (2001) 

9. Hendricks, E., Jannerup, O., S0rensen, PH.: Linear Systems Control: Deterministic and 
Stochastic Methods. Springer, Berlin (2008) 

10. Huang, C.S.: Structural identification from ambient vibration using the multivariate AR 
model. Journal of Sound and Vibration 241(3), 337-359 (2001) 

11. Koulocheris, D.V., Dertimanis, V.K., Spentzas, C.N.: Parametric identification of vehicle 
structural characteristics. Forschung im Ingenieurwesen 72(1), 39-51 (2008) 

12. Landau, I.D., Zito, G.: Digital Control Systems: Design, Identification and Implementation. 
Springer, London (2006) 

13. Lardies, J.: Relationship between state-space and ARMAV approaches to modal parameter 
identification. Mechanical Systems and Signal Processing 22(3), 611-616 (2008) 

14. Lee, J.E., Fassois, S.D.: On the problem of stochastic experimental modal analysis based 
on multiple-excitation multiple-response data, Part I: Dispersion analysis of continuous-time 
systems. Journal of Sound and Vibration 161(1), 57-87 (1993) 

15. Ljung, L.: System Identification: Theory for the User, 2nd edn. Prentice-Hall, Inc, New Jer- 
sey (1999) 

16. Lutkepohl, H.: New Introduction to Multiple Time Series Analysis. Springer, Berlin (2005) 

17. Meyer, CD.: Matrix Analysis and Applied Linear Algebra. Society for Industrial and Ap- 
plied Mathematics, Philadelphia (2000) 

18. Papakos, V, Fassois, S.D.: Multichannel identification of aircraft skeleton structures under 
unobservable excitations: A vector AR/ARMA framework. Mechanical Systems and Signal 
Processing 17(6), 1271-1290 (2003) 



Strain Analysis of the Sciara del Fuoco 
(Stromboli Volcano) 

G. Nunnari 1 , A. Spata 1 , G. Puglisi 2 , A. Bonforte 2 , and F. Guglielmino 2 

1 Universita degli Studi di Catania, viale A. Doria 6, 95125 Catania, Italy 

2 Istituto Nazionale di Geonsica e Vulcanologia - Sezione di Catania 

Piazza Roma 2, 95125 Catania, Italy 

gnunnari@diees . unict . it, alessandro_spata@yahoo . it 
{puglisi-g, bonforte, guglielmino}@ct . ingv. it 



Abstract. In this paper we describe the pre-processing techniques and the the- 
ory of classical mechanics we have adopted in order to computed both 3D mo- 
tion maps and 3D strain tensor over the Sciara del Fuoco (SdF), the Stromboli 
volcano (Italy) steep flank subjects to dangerous landslide events. Movements af- 
fecting the SdF are measured by the monitoring system known as THEODOROS 
(THEOdolite and Distancemeter Robot Observatory of Stromboli). 



1 Introduction 

Stromboli is an active volcano, about 2500 m high above the sea floor. Roughly only 
the last kilometre of this volcano emerges from the sea, forming an island whose di- 
ameter ranges from 2.4 to 5 km. It belongs to the Aeolian Islands and represents the 
most active volcano of this archipelago. Its conic shape is evidently characterized by 
a big depression that marks the northwestern flank of the edifice: the Sciara del Fuoco 
(SDF). On December 28th, 2002, lava flows outpoured from the northern wall of the 
NE crater and descended into the Sciara del Fuoco, a deep depression marking the NW 
flank of the volcano edifice. On December 30th, 2002, two landslides occurred on the 
northern part of the Sciara del Fuoco; they moved a mass in the order of tens of mil- 
lions of cubic meters both above and below sea level. The landslide produced a tsunami 
causing significant damage to the eastern coast of the island, reaching the other Aeo- 
lian Islands and the Sicilian and southern Italian coasts. This event led to the upgrading 
of the ground deformation monitoring system, already existing on the island; the new 
requirement was the real-time detection of the deformations related to potential slope 
failures of the SdF. To this purpose the chosen instrument was the Leica TCA 2003 To- 
tal Station (TS) equipped with GeoMos software [4] that allows remote sensor control. 
The acronym of this system is THEODOROS (THEOdolite and Distance-meter Robot 
Observatory of Stromboli) [7]. The rest of this paper is organized in the following way. 
In Sec. II a brief description of the current THEODOROS configuration is given, the 
interested reader can found more detailed information in [4,7]. Sec. Ill reports the ap- 
proach adopted to pre-processing data; Sec. IV shows the methodology used to compute 
the strain field; Sec. V reports the case study; finally Sec. VI draws the conclusions of 
this study. 



-323. 



J. A. Cetto et al. (Eds.): Informatics in Control Automation and Robotics, LNEE 85, pp. 317 
springerlink.com © Springer- Verlag Berlin Heidelberg 201 1 



318 G. Nunnari et al. 

2 Brief Introduction to the THEODOROS System 

The THEODOROS system consists of a remote-controlled Total Station that can be pro- 
grammed to measure slope distances and angles between the sensor and benchmarks 
appropriately installed in the SDF area at a specific sampling rate. To ensure a con- 
tinuous stream of data from the instrument, it requires a constant power supply and a 
continuous link with the PC controlling the Total Station's activities, installed on the S. 
Vincenzo Observatory, where the National Department of Civil Protection (DPC) con- 
trol room is located. The Stromboli volcano eruption of the 27 February 2007 destroyed 
the THEODOROS benchmarks inside the SDF. A new configuration was designed and 
new benchmarks were installed on the new fan produced by the lava flow entering the 
sea. This new topology consists of six reflectors installed outside the SdF, around the 
Total Station, for the reference system and atmospheric corrections (SENT, BORD, 
SEMF, SPLB2, CIST and EL1S), nine reflectors for monitoring movements of the lava 
fan inside the SdF (SDF18, SDF19, SDF20, SDF21, SDF22, SDF23, SDF24, SDF25 
and SDF26), two reflectors to monitor the northern border of the SdF (400 and BASTI) 
and two further reflectors on stable sites to check the stability of the measurements both 
on short and very long distance measurements (CURV and CRV). Currently the reflec- 
tors SDF20 and SDF21 are not working. The sample time indicated as t c hereafter is set 
to be t c = 10 minutes. Each measurement for each target or reference point provides 
the instantaneous values of three relevant pieces of information: the slope distance (sd), 
the horizontal (hz) angle and the vertical angle (ve). Starting from this information, 
the GeoMos system is able to transform the TS measurement vectors (whose compo- 
nents are sd, hz, ve) into an equivalent vector whose components are expressed in terms 
of North (N), South (S) and Up (U) with respect to the assumed reference system. In 
this computation, GeoMos is able to take into account the constraints imposed by the 
assumption of the reference system. Despite the availability of real-time information, 
this is not enough to automatically evaluate the state of ground deformation. Indeed the 
acquired measures are affected by offsets, spikes and noise sources that strongly com- 
promised their interpretation. These drawbacks must be necessary overcome before that 
suitable quantities related to the ground deformation dynamic can be efficiently com- 
puted. In particular in this paper we focus our attention on the problems of offsets and 
spikes removal, smoothing noisy data and strain tensor evaluation. 

3 Pre-processing Data 

The algorithm we propose to remove both spikes and offsets consists of two steps. 
First the spikes are removed, then attention is focused on offsets. Since the single dis- 
placement components (North, East, Up) of each benchmark in the period June 2006 - 
December 2008 are characterized by a normal distribution, the problem to remove the 
spikes affecting observations, i.e. the sharp variations of the time series which are gen- 
erally due to either periodical maintenance or instrument malfunctions, is well solved 
adopting the standard deviation of observations as reference. Indeed let TsdF^ if) be 
a generic component of the benchmark SdF x at time t, let ATsdF^ if) be the differ- 
ence between two subsequent measures and denoting as a its standard deviation, the 
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experience gained through the daily monitoring of the SdF suggest us to consider as 
spikes the ATsdF^ (t) values falling outside the range covered by one a. 

The offsets affecting observations are essentially due to the maintenances of the 
THEODORO system. Here it is necessary to distinguish two types of maintenances: pe- 
riodical maintenance usually carried out every six months, and extra maintenance due 
to unexpected crash of the system. The offsets related to the periodical maintenance are 
simply adjusted taking into account the marked sharp variation (jump) visible when the 
system begins to work. This approach is also suitable for offset due to the crash of the 
system if the normal functioning of the system is promptly restored. Instead, if the ex- 
tra maintenance is performed after a sufficiently long time the system crashed, then the 
offsets removal is not trivial. Indeed, in this case, in order to perform a reliable offsets 
correction the estimation of the trend of each ground deformation component during 
the period in which the system was crashed is needed. In order to adjust these kinds of 
offsets we use the linear trend as shown in figure 1 . 




Fig. 1. Offsets correction approach based on linear trend 



Although both spikes and offsets removal makes ground deformations more read- 
able, further processing is needed to reduce noise source affecting data, in particular the 
thermoelastic effects on ground deformation due to the temperature. To this purpose we 
have smoothed noisy data with spline functions [1]. A spline function s(t) is a function 
defined piecewise by polynomials. This function takes values from an interval [a, b] and 
maps them to R, the set of real numbers. The interval [a, b] is divided into k disjoint 
subintervals [ti,ti + i] with < i < k — 1 such that [a, b] = [to,ti]U...U[tk-2,tk-i]- 
The given k points ti are called knots. The vector t — (to, .,,tk-i) is called a knot 
vector for the spline. If the knots are equidistantly distributed in the interval [a, b] the 
spline is uniform, otherwise it is non-uniform. On each of this subintervals a nth poly- 
nomial is defined and joined with others polynomials at the knot points in such a way 
that all derivatives up to the (n — l)th degree are continuous. Within these constraints, 
the function s(t) is selected which minimizes: 
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Fig. 2. (a) Original SDF26 North component; (b) Spikes and offset removed; (c) Smoothing noise 



E 



(s(U)- Xi f +p l(s^(t)) 2 dt 



(1) 



where (ti,Xi) are the raw data samples and s(k) denotes the kth derivative of s(t). The 
weight factor p is the smoothing parameter whose value must be opportunely chosen to 
obtain a good compromise between good fit and the smoothness. In figure 2 are shown, 
respectively, the raw data of the benchmark SdF26 (North component), the data after 
removing spikes and offsets and finally the spline-smoothing. 



4 Strain Interpolation 

In order to compute both 3D displacements map and strain tensor in the area of SdF cov- 
ered by the THEODOROS system we use the modified least-square approach introduced 
by [8] and also used by [5] and [9]. Given a point P having position a;o = (xio, ^20 , £30) 
surrounded by N experimental points (EPs) whose positions and displacements are re- 
spectively x(n) = (sci( n ) , x 2 ( n ) , x 3 { n ) ) and u(n) = (u 1{n) , w 2 ( n ) , «3(n)) where n = l..N, 
the problem of estimating both the displacements gradient tensor H and the displacement 
components Ui(i = 1..3) of the point P, according to the infinitesimal strain theory, can 
be modelled in terms of the following strain interpolation equations: 



i(n)(x) = HijAXj^n) + Ui (i,j = 1..3) 



(2) 
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where Axj( n ) — %j(n) ~ x jo are the relative positions of the nth EP experimental 
points and the arbitrary point P and Hij = ^- are the elements of the displacement 
gradient tensor. It can be decomposed in a symmetric and an an ti- symmetric part as 
H = E + Q, where E is the strain tensor defined as: 

E=-{H ij +H ji )e i ®e j (3) 

and J? is the rigid body rotation tensor defined as: 

Q = -{H i j-H ii )e i ®e i (4) 

Here ej is the base vector of the Cartesian reference system and © is the tensor product. 
In a compact form the undetermined system of equations (2) can be written as Al = u 
where A is the design matrix simply derivable from equation (2), I = 
[U\ XJ<l U3 en ei2 £13 £22 £23 £33 u>i ^2 W3] is the vector of unknown parameters and 
u = [u(l) u(2) u(n)] T is the observation vector. Assuming a uniform strain field and 
re-writing the previous linear equation (4) as Al = u + e, where e is the residual vector 
which model the stochastic nature of the estimation problem, a suitable method to solve 
the system is the Weighted Least Squares (WLS) which gives the expression (5) as a 
suitable formula to estimate the unknown vector I 

l = (A T WA)- 1 A T Wu (5) 

In (5) W is the data covariance matrix. Usually W is assumed to be diagonal (uncorre- 
cted data), i.e. of the form 



W = 



diag(a- {1) ,a- (1) ,a- (1) ,...,a- {n) ,a- {n) ,a- (n) ) (6) 



where the quantities (Jj( n ) s are the standard deviations of the measurements. According 
to the modified least squares (MLS) approach proposed by [8], based on the adjustment 
of the covariance matrix W, we use the matrix W which is a weighted version of 
the matrix W of experimental data. Following the suggestion given by [8] and [9] the 
weight function considered here is: 

W' = We'^T (7) 

where dt n ) is the distance between the nth EP and the arbitrary point P, and do is a 
distance-decaying constant defining the "level of locality" of the estimation. 

This method, likewise most previous methods [2] and [6] is used to interpolate the 
strain among benchmarks of geodetic networks where ground deformations are mea- 
sured by comparing geodetic surveys. 
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Fig. 3. (a) In the frames (a), (b) and (c) are reported the calculated East, North and Up components 
of displacements respectively. Frames (d) and (e) report the maximum shear strain and the volume 
variation. Finally, in the frame (f) the displacement vectors of the benchmarks are shown. 



5 An Application to the Sciara del Fuoco 

The benchmarks placed on the lava fan show a general NW-ward motion following the 
maximum slope of the SdF, with an increasing magnitude from NE to SW (Fig. 3). 
This kind of deformation is in good agreement with a seawards motion of the new lava 
fan, driven by a mainly gravitational dynamics. However, the ground motion is not uni- 
form above the investigated area, showing significant differences in the displacements 
measured on different benchmarks. In order to analyze the ground deformation pattern 
recorded from December 14, 2008 to January 3, 2009 above the deforming lava body, 
we performed a strain interpolation. Unfortunatly the corresponding linear system is 
undetermined since it implies more unknows (n — 9) that equations (to = 7). There- 
fore the solution is never unique. To this reason we have calculated a basic solution with 
almost to non zero components by using a MATLAB algorithm based on the QR factor- 
ization with column pivoting. Results are reported in Fig. 3, where the decrease of the 
horizontal motion (Fig. 3a and b) is evident from benchmark SdF25, that is located on 
the upper and central area of the fan, towards the coastline and towards North, reaching 
the minimum values at SdF18 benchmark, located close to the SdF northern rim. The 
vertical motion (Fig. 3c) shows a more uniform gradient, from a maximum down-lift of 
about 50 mm at the S-Westernmost benchmark (SdF22) to near at SdF18. A deeper 
analysis of the overall deformation of the lava fan is allowed by the interpolation of the 
strain tensor. In Fig. 3d, the distribution of the maximum 3D shear strain is reported, 
confirming the strongest deformation on the upper area of the lava fan; this is mainly 
due to the stronger magnitude of horizontal displacements of the southernmost SdFS22, 
SdFS25 and SdFS26 benchmarks with respect to the northern half of the fan, but also 
to the relative vertical motion of the two halves of the body. On the upper area, also the 
volumetric dilatation evidences a maximum expansion (Fig. 3e), mainly imputable to 
the divergent directions of the displacements affecting SdF25 and SdFS26 benchmarks. 
In addition, a contracting area is detected on the southern coastline of the fan, due to the 
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smaller displacements of the SdF23 and SdF24 benchmarks with respect to the upper 
ones, while all the northern half of the lava body shows no significant volumetric strain 
variation confirming the higher stability of this portion of the fan that is buttressed by 
the stable northern wall of the SdF. 

6 Conclusions 

In this paper we have first shown the pre-processing techniques adopted to reduce noise 
sources affecting ground deformation measures acquired at the Sciara del Fuoco by 
the automatic monitoring system referred to as THEODOROS. In particular, due to 
the gaussian distribution of acquisitions, the problem of spikes removal was simply 
solved taking into account their standard deviations. The offsets due to the crash of the 
system have been adjusted based on the evaluation of the linear trend of observations. 
Finally spline functions have been used to reduce the thermoelastic effects. After these 
pre-processing steps we have shown the based on infinitesimal strain theory method 
used to compute both displacements maps and strain field over the area covered by the 
THEODOROS system. Finally a case study related to the ground motion observed in 
the period December 2008 - January 2009 was carried out in order to test the proposed 
methodology. 
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Abstract. The paper presents a novel modification to the generalised condition- 
ing technique anti-windup compensator presented as a triplet of its possible mod- 
ifications. The results are presented from application of pole-placement control, 
with an analysis of closed-loop system behaviour for a class of second-order min- 
imumphase stable plants of oscillatory and aperiodic characteristics with different 
dead-times. 



1 Introduction 

Constraints are ubiquitous in real-world environment. As the result of their presence or 
the presence of some nonlinearities in the control loops, arises the difference in between 
computed and applied (i.e. constrained) control signal. In such a case, the performance 
of the closed-loop system degrades in comparison with the performance of the linear 
system, when constraints are not active. Such a degradation is denned as windup phe- 
nomenon 1415161 . 

This can be also viewed from the point of discrepancy in between internal controller 
states and its output. When there is no correspondence in between controller's output 
and its internal controller states, the controller does not have any information what the 
current value of the constrained control signal is, and windup phenomenon arises. 

The windup phenomenon has been, at first, defined for controllers comprising in- 
tegral terms, as the so-called integrator windup [4|. For such controllers, control con- 
straints may cause excessive integration of the error signal, giving rise to longer settling 
of the output signal and overshoots. There are two strands in compensating windup phe- 
nomenon (in AWC, anti-windup compensation) - taking constraints into account dur- 
ing the design procedure of the controller or assuming the system is linear, designing 
the controller for the linear case, and, subsequently, imposing constraints and applying 
AWCs JT12I31 . 

The simplest anti-windup compensators have been based on the idea of integrator 
clamping, i.e. they referred to the controllers comprising integral terms only [7|. The 
proposed AWCs avoided integration of the error signal whenever some conditions were 
met, e.g., the control signal saturated, or error exceeded some predefined threshold, etc. 

Such an approach was simple enough to be easily implemented, but as it has been 
already said, applicable to some controllers only. 

J. A. Cetto et al. (Eds.): Informatics in Control Automation and Robotics, LNEE 85, pp. 325 |-336.| 
springerlink.com © Springer- Verlag Berlin Heidelberg 201 1 
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The advanced anti-windup compensators have been designed for the case of general 
controller, which input-output equation is written in the RST form. Among the proposed 
AWCs one can find in the literature deadbeat, generalised, conditioning technique, 
modified conditioning technique and generalised conditioning technique anti-windup 
compensators [2 3|. The three latter AWCs are based on the idea of back-calculation, 
i.e. modification of the signal that the output signal of the plant is to track, with respect 
to current saturation level. 

The paper focuses on the generalised conditioning technique AWC (GCT-AWC), 
being a compromise solution in between the simplicity of the advanced AWC and com- 
pensation capabilities of the conditioning algorithm, what will be explained later. 

The main idea of the paper is to present a modification of the GCT-AWC that can 
arise from the idea of integrator clamping methods, and to show that it can result in 
better control performance than performance of the system with original GCT-AWC. 
The presented results refer to the research carried for a set of stable minimumphase 
second-order discrete-time plants and different constraint levels. 

There are no remarks in the literature how to improve the performance of the GCT- 
AWC, apart from [2|. The proposed method limits the number of modifications, with 
the same excess. By introducing the proposed modifications one can improve the per- 
formance of the most appealing AWC technique. 



2 Plant Model 

Let the discrete-time CARMA model be given 

A{q- l )y t = B{q- x )ut- d , (1) 

where y t is the plant output, Ut is the constrained control input, d > 1 is a dead-time 
and: 

A(q- 1 ) = l + a 1 q- 1 +a 2 q- 2 , (2) 

B{q- l ) = b + b iq - 1 (3) 

are relatively prime. The control signal Ut — sat(i>t; a) is the computed control signal 
after saturation by symmetrical cut-off function at level ±a. 

3 Controller 

The plant is controlled by the pole-placement controller that ensures tracking of a given 
reference signal r t by the plant output y t with given dynamics, 

q- 1 



Vt = k R r t - k P y t + fc/- r (r t - y t ) 

i-cr 1 

1-79 



k D-, ±ZiVt, (4) 
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where kp — rkp, r > 0. The above controller equation can be obtained by discreti- 
sation of a continuous-time PID controller [4], and it can be rewritten into the RST 
structure 

R(q- 1 )v t = -S(q- 1 )y t +T(q- 1 )r t . (5) 

Coefficients of polynomials R(q^ 1 ), S(q~ 1 ), T(q~ 1 ) can be determined by solving 
the following Diophantine equation 

A(q- 1 )R( q - 1 )+q- d B(q- 1 )S(q- 1 ) = 

= AMiq-^Aoiq- 1 ) , (6) 

where polynomials A (q^ 1 ) and A^iiq" 1 ) are stable, and given polynomial Am(§ _1 ) 
is of second order in the chapter. 

Controller polynomials R(q^ 1 ), S(q^ 1 ), T(q^ 1 ) are of order d + nB, nA, nA , 
respectively, and have forms as follows: 



(7) 



From the controller equation (0, given structure R(q 1 ), 5(q 1 ), T(q x ) (Q and © 
it follows that: 



R(q- 


^ 


-(1-8- 1 


)R'(q' r ), 


S(q- 


' X ) = 


= s + sxq 


- 1 +S2q- 2 


T(q- 


- 1 )- 


= k R A (q~ 


- 1 )- 



so 


= k P + k D , 


Si 


= k I -2k D -k P (l+ 1 ) 1 


S2 


= k D -j(ki - k P ) , 


- 1 ) 


= (1-jq- 1 ) (l-q- 1 (l 




= 1 + a ol q~ 1 +a o2 q~ 2 , 



(8) 



(9) 

where 7= ~^,k R = rk P ,a ol = £^-(l+7),a o2 = 7(1- |^ J . As the polynomial 

A (q~ 1 ) has to be Stable, < |p- < 2 must hold what can be ensured by a proper choice 
of r. 

The controller algorithm is assumed to be altered by anti-windup compensator pre- 
sented in the next Section, in order to assure better control performance of the closed- 
loop system subject to constraints. It is to be borne in mind that the compensation is 
based on back-calculation, i.e., it does not require the controller to have integral terms 
in general. 



4 Generalised Conditioning Technique AWC 

In GCT, the filtered set-point signal is conditioned, instead of the set-point r t , and given 
as 

QOr 1 )!^- 1 ) 

''■« = L(q-i) n > (10) 
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with 



T(q~ 1 )=T 2 (q- 1 )T 1 (q- 1 ), 

Qiq- 1 ) = 90 + 9KT 1 + • ' • + q nQ q-' lQ , UD 

L(q- 1 ) = l + l 1 q- 1 + --- + l nL q- 



n L 



andT 2 (0)=t 2 ,o. 

Similarly to the conditioning method (see [2Q, the modified filtered reference signal 
is given by 

r f,t = rf, t + , (12) 

12,0 

and the control signal is defined as 

Vt = (1 - Q '(q-^Riq- 1 ))^ + ^£ r/ + 

<7o 

+ L((T 2 (q- 1 )L(q- 1 )-t 2fl )rl t - 
<7o 

-Q'{q- l )S(q- l ) Vu (13) 

whereg'(g- 1 ) = ^p. 

The GCT method enables additional tuning of the performance by reference signal 
filter design. Because its parameters should correspond to model parameters, saturation 
level and set-point values, a special choice of parameters of the filter < TT0b for mini- 
mumphase second-order model is proposed [2|. Let gi and g 2 denote poles of stable 
A(z^ r ), then 

g = max(|£>i|, \g 2 \) , (14) 

Qiq- 1 ) = 1 + ((1 - e )« - 1) q- 1 , (15) 

L(q- 1 ) = 1 - (1 - gfq- 1 , (16) 

where < £ < 1 is the damping factor obtained from classical root locus theory for 
the second-order systems. The suggested filter ([T4HT6l l takes into consideration model 
parameters and set-point values only, forcing the initial values of the filtered reference 
signal for slow models and reducing the amplitude and rate of transients for oscillatory 
ones., see FigureQ] 

The inherent property of the conditioning technique is the so-called short sightedness 
phenomenon, resulting in consecutive resaturations of the control signal because of 
excessive modification of the reference signal. In order to improve the performance of 
the compensation three modifications will be considered as in the next Section. 

5 Modified Generalised Conditioning AWCs 

In order to combine classical conditional integration methods that work for controllers 
comprising integrators with back-calculation AWC presented in the previous Section, 
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Fig. 1. Plant step response h t and reference filter step response r/,t for a) oscillatory and b) 
aperiodic plant 

the following three back-calculation modifications have been proposed - the modifica- 
tion of the filtered reference input is applied when: 

Ml |e t |>ei, 

M2 u t -i ^ v t -i , 

M3 Ut-i 7^ vt-i and e t u t -\ > , 
where ei is a threshold value for reference modification clamping. 

By applying the modifications to the GCT-AWC one assures that modification of the 
filtered reference signal is performed only when necessary. 

6 Simulated Plants 



The simulations have been performed for a set of stable, second-order, minimumphase 
plants with B(q^ 1 ) = 1 + 0.5q _1 and: 



- PI type 



Aiq- 1 ) = (1 - q-\a + u»))(l - q-\a - ui)) . 
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where: 

-1 < a < 1, 

-1 < u < 1, 

<7 ± Wt| < 1 , 

what corresponds to oscillatory behaviour of the plant, 
- P2 type 

A(q- 1 ) = (l-q- 1 zi)(l-q- l Z 2 ), 

where: 

< z\ < 1 , 

< z 2 < 1 , 

what corresponds to aperiodic behaviour of the plant. 

The simulations have been run for square wave reference signal of period 40 samples 
and symmetrical amplitude ±3 with |^- = 0.5, A M (q~ l ) = 1 - 0.5q _1 + 0.06q~ 2 and 
ei = 3. 

In order to evaluate the quality of regulation process, the performance index is intro- 
duced 

J = ^J2l (r t -y t ) 2 , (17) 

where N = 150 denotes the simulation horizon. 

The simulations have been performed for the same constraint hardness for each of 
the plants, denoted by relative constraint level a r (i.e., the multiplicity of the minimum 
constraint level a m \ n = 3 LU allowing asymptotic tracking). The absolute value of 
the constraint is a — a r a m ; n and changes as the plants change. 



7 Performance Surfaces 

The results of the simulation tests are shown as performance surfaces. Each of the axes 
has been divided into 101 values, thus all simulation results refer to a grid of 101 x 101 
different plants. The idea of such surfaces is as follows - let Jo denote the value of the 
performance index of the control system with some plant and given a r and no AWC. 
Let J\ denote the value of the performance index of the same control system with the 
same plant but with classical GCT-AWC. Let Ji denote the value of the performance 
index of, again, the same control system with the same plant but with modified GCT- 
AWC (Ml, M2 or M3). 

For each of the plants and constraints level the following face is plotted: 

■ (magenta) J = Ji = J 2 , 

■ (red) modification is of the worst performance, Jo < J\ < Ji or J\ < Jo < J2, 
the intensity of the red level is proportional to J 2 — Jo or J2 — J\ , 

□ (white) modification improves the performance of the GCT-AWC, Jo < J2 < Ji, 
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■ (black) it is not worth to modify GCT, J\ < J 2 < Jo, the intensity of the black 
level is proportional to J2 — Ji , 

■ (blue) modification improves the performance where GCT fails to, J2 < Jq < Ji, 
the intensity of the blue level is proportional to Jq — J 2, 

(green) modification is of the best performance, the intensity of the green level is 
proportional to Jo — J2 or J\ — J 2 ■ 

8 Should One Modify GCT? 

The performance surfaces have been obtained for PI and P2 type plants with different 
dead-times and presented in Figs 1-4, where consecutive rows for different dead-times 
refer to Ml, M2 and M3. 

In all the cases of PI and d = 1 it is visible that all modifications can improve the 
performance of the GCT for slow plants, i.e., with small natural frequency, whereas in 
the case of Ml and M3 there is an improvement visible for such plants near stability 
border. In the case of Ml and M3, one can see region of the best improvement (green). 
By comparing the given surfaces one can say that M3 is of the best AWC performance, 
because of the green regions and brighter red regions than in other cases, what refers to 
less performance degradation. 

It is not advisable to modify the GCT algorithm when the region is red, it is advisable 
to improve where it is white and definitely advisable when green. 

In the case of d — 3 one can see that red regions have almost disappeared and the 
improvement is best in the case of M3. 

For P2 type plants a performance improvement can be observed for slow plants 
(green) with Ml and M3. Because of the size of white and green regions one can say 
that the best performance is assured by Ml, mainly because of the a r > 1, that is vis- 
ibility of green regions for greater a r s. The vast areas of red color suggest that it is 
inadvisable to modify the original GCT when plant is moderately slow (expressed by 
absolute values of its poles). 

In the case of d = 3 because of the area of white region and brightness of the red 
region, it can be said that Ml is the best choice, then M2 and M3. 

9 Summary 

It has been shown that it can be advantageous to modify the algorithm of well-known 
GCT-AWC in order to obtain high control performance. Such a modification can be 
implemented with the use of lookup table, where the information is stored what GCT 
algorithm should be used when plant parameters vary in time, e.g. due to aging or 
set-point change. A similar approach has been presented for continuous system, PID 
controllers and integrator clamping [7|. 
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Fig. 2. Performance surfaces for PI, d = 1 
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Fig. 3. Performance surfaces for PI, d = 3 
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Fig. 4. Performance surfaces for P2, d = 1 
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Fig. 5. Performance surfaces for P2, d = 3 
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Abstract. The autonomous systems that operate in physical, random and highly 
dynamical environments involve a large amount of computation, which depends 
on the number of parameters. We tackle the problem of complexity reduction for 
these systems based on an interdisciplinary and qualitative approach. We define 
concepts like qualitative model reduction and adaptive bismulation, and use them 
to investigate the stochastic model checking. These concepts prove to be very use- 
ful in capturing the co-evolution between system and its environment. Potential 
applications in renewable energies are discussed. 

Keywords: Autonomous systems, Uncertain cyber-physical systems, Adaptive 
bisimulation, System environment co-evolution, Stochastic model checking, Re- 
newable energy. 



1 Introduction 

System autonomy is becoming a crucial feature of the modern systems. For systems 
deployed in remote locations like wind turbines, or space systems, the processes of 
monitoring and control are increasingly difficult to execute. For such systems, nowa- 
days called cyber physical systems, the autonomic capacity of self monitoring, self op- 
timizing and self healing is essential. A major problem in achieving autonomy is raised 
by the massive amount of computation necessary to evaluate various (optimization, per- 
formance or safety) criteria. In most cases, the computation exponentially depends on 
the number of parameters. Therefore, reducing the state space size for reasoning about 
the system behaviour represents a major research topic for autonomous cyber physical 
systems. 

Currently, in the research community, there are several concurrent definitions for 
cyber physical systems. For our purposes, a cyber physical system involves a tight in- 
teraction between physical and computational processes [14]. This interaction can be 
due to the presence of heterogeneous discrete/continuous components (like the case 
of hybrid systems) or because of the system deployment within an uncertain dynamic 
physical environment (like the case of embedded systems). 

In this paper, we introduce a general model for uncertain cyber physical systems 
operating in a random dynamical environment. In this model, the environment influ- 
ence on the system is modelled implicitly using stochastic differential equations. The 
computational processes are modelled in a computer science fashion, as sequences of 
discrete transitions. The interaction between physical and computational processes is 
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captured by interleaving using stochastic kernels. For this general model, we define an 
abstract measure for performance/safety in the form of a probability of reaching a given 
target state set. The evaluation of this measure is called probabilistic model checking 
in computer science and stochastic reachability analysis in control engineering. For the 
general model we consider, this evaluation will be called stochastic model checking. 

For stochastic model checking, efficient computational methods exist only for certain 
classes of systems. For uncertain cyber physical systems, obtaining efficient evaluation 
methods is still an open problem. This situation is also due to underdeveloped mathe- 
matical investigation of these systems. In this work, we address this problem combining 
tools from mathematics and computer science in the form of Hilbertian formal meth- 
ods, which is a recent modelling paradigm based on functional analysis. We present a 
method for reducing the size of the state space up to a single dimension, if necessary. 
This method resembles of model order reduction from numerical analysis, but it is casted 
in a qualitative manner. This is why, the method is called qualitative model reduction. 
As an application, we define adaptive bisimulation which is a behavioural equivalence 
concept. Another application is to model co-evolution between the system and its envi- 
ronment and we show the co-evolution is preserved by the qualitative model reduction. 

In the end, we propose an application area drawn from renewable energy industries. 
Specifically, wind turbines are uncertain cyber physical systems that closely interact 
with a very dynamic physical environment. For wind turbines, it is desirable to have 
autonomous controllers that self optimize the turbine performance according with the 
changes within environment. 

2 The Mathematical Setting 

2.1 Uncertain Cyber-physical Systems 

The theory of hybrid systems is a well-established modeling paradigm for embedded 
systems. Similarly, the theory of concurrent embedded hybrid systems [3| constitutes 
a suitable modeling framework for CPS. In the following an uncertain cyber-physical 
system is modeled as a randomized embedded hybrid system. 

There are two major ways to randomize a continuous or hybrid dynamical system: 
In one approach, the concept of noise is used to model small random perturbations. 
The randomized system has trajectories that closely resemble those of the deterministic 
initial system. The noise based randomization is carried out using stochastic differential 
equations. When the influence of the random perturbation changes dramatically the 
system evolution, the randomization is carried out using stochastic kernels that replace 
the concept of reset maps from deterministic hybrid system models. 

A UCPS U = (Q, X, F, R, A) consists of 

• a finite set of discrete variables Q; 

• a map X : Q — > M. d (-> that sends each q <E Q into a mode (an open subset) X q of 
R d (9) t where d(q) is the Euclidean dimension of the corresponding mode; 

• a map F : Q — > 2 :FsDE which specifies the continuous evolution of the automaton in 
terms of stochastic differential equations (SDE) over the continuous state x q for each 
mode; 
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• a family of stochastic kernels R = (R q ) q< zQ, 

R":X q x (US(X')|j £ Q\{q}) - [0, 1]; (1) 



• a transition rate function 



X:(UX J \jeQ)^R + , (2) 



which gives the distributions of the jump times. 

The executions of a UCPS can be described as follows: start with an initial point 
xo £ X q , follow a solution of the SDE associated to X q , jump when this trajectory hits 
the boundary or according with the transition rate A (the jump time is the minimum of 
the boundary hitting time and the time, which is exponentially distributed with the tran- 
sition rate A). Under standard assumptions, for each initial condition x € j £ QUX^, 
the possible trajectories starting from x, form a stochastic process. Moreover, for all ini- 
tial conditions x, the executions of a UCPS form the semantics, which can be thought 
of as a Markov process in a general setting. Let us consider M = (J?, !F, Fti%tiPx) be 
the semantics of U. Under mild assumptions on the parameters of U, M can be viewed 
as a family of Markov processes with the state space (X, B), where X is the union of 
modes and B is its Borel cr-algebra. Let B b (X) be the lattice of bounded positive mea- 
surable functions on X. The meaning of the elements of M can be found in any source 
treating continuous-parameter Markov processes (see, for example, |9|). Suppose we 
have given a cr-finite measure \i on (X, B). 

In the following we give some operator characterizations of stochastic processes, 
which are employed in this paper to define a qualitative model reduction for UCPS. 

2.2 Hilbertean Formal Methods 

The HFM [4| abstract away the analytical properties of deterministic and stochastic 
differential operators using the so called kernel operator (defined in the following). Us- 
ing methods of functional analysis HFM elegantly generalize both deterministic and 
stochastic systems. In this work we focus on the stochastic case. Let us describe briefly 
the mathematical apparatus that is usually employed to study continuous time continu- 
ous space Markov processes. 

The transition probability function is p t (x,A) — P x (x t £ A), A £ B. This is 
the probability that, if xo = x, Xt will lie in the set A. The operator semigroup V is 
defined by P t f{x) = J f(y)pt(x,dy) — E x f(xt),Vx £ X, where E x is the expec- 
tation w.r.t. P x . The operator resolvent V = {V a ) a >o associated with V is V a f(x) = 
L e~ at Ptf(x)dt, x £ X. Let denote by V the initial operator Vq of V, which is known 
as the kernel operator of the Markov process M. The operator resolvent (V a ) a >o is the 
Laplace transform of the semigroup. 

The strong generator/^ is the derivative of P t att = 0. Let D(C) C Bb(X) be the set 
of functions / for which the following limit exists (denoted by £/): lim^o j (Ptf — /) • 

In the HFM, there is developed a semantic framework for concurrent embedded sys- 
tems constructed using energy forms. We specialize this theory for function spaces, 
reaching in this way the theory of Dirichlet forms lT2ll . 
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A quadratic form £ can be associated to the generator of a Markov process in a 
natural way. 

Let L 2 (X, /i) be the space of square integrable /i-measurable extended real valued 
functions on X, w.r.t. the natural inner product < /, g > /1 = J f(x)g(x)dfi(x). 
The quadratic form £: 

£(f,g) =-<Cf,g> fi ,f£ D(C),g e L 2 (X,n) (3) 

defines a closed form. This leads to another way of parameterizing Markov processes. 
Instead of writing down a generator one starts with a quadratic form. As in the case of a 
generator it is typically not easy to fully characterize the domain of the quadratic form. 
For this reason one starts by defining a quadratic form on a smaller space and showing 
that it can be extended to a closed form in subset of L 2 (fi). When the Markov process 
can be initialized to be stationary, the measure /i is typically this stationary distribution 
(see (9j p.lll). More generally, /i does not have to be a finite measure. 

A coercive closed form is a quadratic form (£ , D{£)) with D(£ ) dense in L 2 (X, //), 
which satisfies the: (i) closeness axiom, i.e. its symmetric part is positive definite and 
closed in L 2 (X, /i), (ii) continuity axiom. £ is called bilinear functional energy (BLFE) 
if, in addition, it satisfies the third axiom: (iii) contraction condition, i.e. Vw <E D{£), 
u* = u + A 1 e D(£) and £{u ± u*,u =F u*) > 0. For the general theory of closed 
forms associated with Markov processes see [ 12|. 

Let (C, D{C)) be the generator of a coercive form (£, D{£)) on L 2 (X, /x), i.e. the 
unique closed linear operator on L 2 (X,n) such that 1 — £ is onto, D(C) C D(£) 
and £(u,v) =< -Cu,v > for all u £ D(C) and v <E D(£). Let (T t ) t>0 be the 
strongly continuous contraction semigroup on L 2 (X, /i) generated by C and (G a ) a> o 
the corresponding strongly continuous contraction semigroup (which exist according to 
the Hille-Yosida theorem). 

A right process M with the state space X is associated with a BLFE (£, D(£)) on 
L 2 {X, ii) if the semigroup (P t ) of the process M is a //-versioiy of the form semigroup 
(T t ). It has been proved [ 1] and [12] that only those BLFEs, which satisfy some reg- 
ularity conditions can be associated with some right Markov processes and viceversa 

(Th.l.9of|[D). 

Prop. 4.2 from [1| states that two right Markov processes M and M' with state 
space X associated with a common quasi-regular BLFE (£, D{£)) are stochastically 
equivalent [12]. That means a quasi-regular BLFE characterizes a class of stochastically 
equivalent right Markov processes. 

Let M — (f2,J 7 ,J-'t,xt,P x ) be a right Markov process with the state space X. 
Now assume that X is a Lusin space (i.e. it is homeomorphic to a Borel subset of a 
compact metric space) and B(X) or B is its Borel er-algebra. Assume also that /x is a 
cr-finite measure on (X, B) and /i is a stationary measure of the process M . Let X# 
another Lusin space (with B# its Borel cr-algebra) and F : X — > X# be a measurable 
function. Let a{F) be the sub-c-algebra of B generated by F. If /i is a probability 
measure then the projection operator between L 2 (X, B, /i) and L 2 (X, cr(F), /i) is the 
conditional expectation £^[-|F]. Recall that E^ is the expectation defined w.r.t. P^ and 



1 I.e., for all / 6 L 2 (X, /i) the function P t f is a /i-version (differs on a set of /i- 
measure zero) of T t f for all t > 0. 
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that P^A) = J P x (A)dfM, A £ F. We denote by fi* the image of \x under F, i.e. 
H#(A#) = fi(F~ 1 (A^)), for all A# £ 23 #. In general, anything associated with X# 
will carry the #- superscript symbol in this section. 

Let £ be the BLFE on L 2 (X, j«) associated to M. F induces a form £ # on L 2 (X # , 
M # )by 

£*{u*,v*) =£(u*oF,v* oF); (4) 

for u#,v# £ D[£ #], where 

L>[£ # ] = {u # eI 2 (I*/i # )|«#oFeD[f]}. (5) 

It can be shown (see Prop. 1.4 in [11 j), under a mild condition on the conditional ex- 
pectation operator E^ [-\F] that £# is a BLFE. If, in addition, £# is quasi-regular then 
we can associate it a right Markov process M# = {O, F, Ft, xf ', Pf) with the state 
space X&. The process M# is called the induced Markov process w.r.t. to the proper 
map F. If the image of M under F is a right Markov process then xf — F(xt). The 
process M# might have some different interpretations like a refinement of discrete tran- 
sitions structure, or an approximation of continuous dynamics or an abstraction of the 
entire process. It is difficult to find a practical condition to impose on F, which would 
guarantee that £&, as defined by (|4| and (0, is also quasi-regular. To circumvent this 
problem, it is possible to restrict the original domain D[£ #] and impose some regularity 
conditions on F (for more details, see [ 1 1 1). 

Assumption 1. Suppose that £ # is a quasi-regular BLFE. 

3 Stochastic Model Checking 

Let us consider M = (f2, F, Ft, x t , P x ) a strong Markov process, which is the seman- 
tics of a UCPS. For this strong Markov process we address a verification problem con- 
sisting of the stochastic reachability problem defined as follows. Given a set A £ B(X) 
and a time horizon T > 0, let us to define |6l : 

Reach T (A) = {uj e Q | 3t G [0,T] : x t (u>) £ A} 

Reach^A) = {uj e ft \3t > : x t (cu) e A}. (6) 

These two sets are the sets of trajectories of M, which reach the set A (the flow that 
enters A) in the interval of time [0, T] or [0, oo). 

The reachability problem consists of determining the probabilities of such sets. The 
reachability problem is well-defined, i.e. ReachT(A), Reach^A) are indeed measur- 
able sets. Then the probabilities of reach events are 

P(T A < T) or P(T A < oo) (7) 

where Ta = inf {t > 0\xt £ ^4} and P is a probability on the measurable space (ft, F) 
of the elementary events associated to M. P can be chosen to be P x (if we want to 
consider the trajectories, which start in x) or P^ (if we want to consider the trajectories, 
which start in xq given by the distribution /i). 
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Usually a target set A in the state space is a level set for a given function F : X — * K, 
i.e. A = {x G X\F(x) > 1} (F can be chosen as the Euclidean norm or as the distance 
to the boundary of E). The probability of the set of trajectories, which hit A until time 
horizon T > can be expressed as 

P(supF(x t )\te[0,T])>l. (8) 

Our goal is to define a new stochastic process M# (qualitative model reduction) such 
that the probabilities (0 are preserved. Ideally, since © can be written as ®, F(xt) 
would represent the best candidate for defining a possible qualitative model reduction 
for M, which preserves the reach set probabilities. The main difficulty is that F(xt) is 
a Markov process only for special choices of F 1 13 1. The problem is how to choose an 
appropriate F. 

Note, if A# is open in X# and A = F~ 1 (A^), then we consider the two first 
hitting times T A (w.r.t. M) and T* # (w.r.t. M#) of A and A*, respectively. Recall that 
T A = inf{i > 0\x t G A}. 

The following results show that the stochastic model checking problem is solvable 
for uncertain cps. 

Proposition 1. Under the assumption^ if [i is a probability measure and £ = +oo (M 
has no killing), then 

F p exp(-T A ) < E^ exp(-Tf # ) (9) 

where E^ (resp. E^#) is the expectation defined w.r.t. P^ (resp. P^#). 
Proof. Using Th. 3 . 1 from [ 1 1 , we have 

Eii# exp(-T* # ) = £*{e A #,e A #)\ E^exp(-T A ) = £ x (e A ,e A ) 

Here, e A # and e A # are the potential and respectively the copotential of A# w.r.t. £# . 
e A # is the 1-reduced of 1 on A#, i.e. is the unique 1-excessive function in D[£#] such 
that e A # = 1 ^ # -a.e. on A* and £f{e A #,v*) = 0, \?v* G D[£*] X# \ A # (the ele- 
ments of D[£#], which take the value on A#). e A # is the 1-coreduced of 1 on A# 
defined correspondingly with the two entries of £* interchanged (see lfT2l . for detailed 
definitions and characterizations). In the same way, e A and e A are defined w.r.t. £. Then 
e A # oF = 1/x-a.e. onF" 1 ^*) and£i(e A # oF,v*oF) = 0, Vw # G D[£*] X# \ A # 
or £i(e A # o F, v) = 0, Vu G D[£] X \ A such that there exists v& G D[£^]x*\A* an d 
d = u*oF. Moreover, it can be easily checked, using (TBI , that e A # o F is 1-excessive 
function in D[£ ]. Then e A # o F might be a candidate for the potential of A w.r.t. £ if 
F is 'compatible' with the cone of excessive functions associated to the given process 
(if Fx = Fx' = x* G X*\A* then v(x) = v(x'),Vv G D[£] X \ A ). We have similar 
relations for e A # . Therefore 

F M #exp(-Tj#) =£*{e A #,e A #) = £ *{e A #, e A #)+ < e A #,e A # > p # 
= £(e A # o F, e A # o F)+ < e A # o F, e A # o F > M = £\{e A * ° F, e A # o F) 
= fi(eA,eA) +£i{e A # a F - e A ,e A # oF-e A ) 
> £i(e A ,e A ) = E^cxp(~T A ) 
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Last inequality is the result of the fact that £ \ (e^* o F — &a-,^a* ° F — ejCj > 0. This 
comes from the characterization of &a as the smallest function, which is equal to 1 on 
A and is bounded by 1 in rest of the space (Prop.l.5| 12|). Then ca < e^# o F, With 
a similar argument we get that ba < &a# ° F . ^ n me symmetric case, the result is a 
consequence of the Th.1.8 from ifTTIl , 

If M is the semantics of a UCPS U, given a target state set A e B(X), then the goal 
in the stochastic reachability analysis is to compute the probability P^ (Ta < T) for a 
finite horizon time T > 0. We now translate the relation (O in terms of probability of 
the reachable sets. 

Proposition 2. Under the assumption^ if fi is a probability measure, then 

Pf,{T A <T)< eKmin{T£*(u#,u#) + < u*,u* > p# \u* e D(£*),u* > 
1, pft — a.e. on A#} 
where K > is the sector constant of£. 



4 Adapting Verification to Co-evolution 

The idea is to apply a "state space reduction" technique based on the general 'induced 
BLFEs' method to achieve qualitative model reductions for UCPS. With this technique, 
the semantics of UCPS are 'approximated' by a one-dimensional stochastic process 
with a much smaller state space. 

4.1 Qualitative Model Reduction 

The stochastic reachability definition gives the idea to introduce the following concept 
of qualitative model reduction (QMR) for UCPS. 

Definition 1. Given a right Markov process M defined on the Lusin state space (X, B), 
and F : X —> WLa measurable weight function, suppose that assumption\l\is fulfilled. 
The process M# associated to the induced BLFE £# under function F is called a QMR 
ofM. 

Let U be a UCPS and M its semantics. Suppose that M is a right Markov process 
defined on the Lusin state space (X, B). 

Definition 2. Any UCPS U# whose semantics is a QMR ofM is called a QMR ofU. 

Let U be a UCPS and M its semantics (that is a right Markov process, with the state 
space X). 

Proposition 3. If M is a diffusion then any QMR M# ofM is a diffusion. 

Proposition 4. If M is a jump process then any QMR M# of M is again a jump pro- 
cess. 
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Proof. This statement can be obtained as a consequence of the abstract version of the 
Kolmogorov backward equations 1 9 1 

^-Ptf(x) = LP t f(x), P f = /, / G D(C) (10) 

at 

and the equality J12I I. If the equations ( fTOb are associated to an initial diffusion process 
(resp. jump process) then the relation ( TT2T > allow to obtain the fact that the transition 
probabilities of the induced process satisfy a similar equation, such that the induced 
process is still a diffusion process (resp. jump process). The same conclusion can be 
obtain using the stochastic calculus of BLFEs IfTTl . ■ 

Since the semantics of a UCPS is, in most cases, a stochastic process, which can be 
viewed an interleaving between some diffusion processes and a jump process (see |5| 
for a very general model for UCPS and its semantics as a Markov string), we can write 
the following result as a corollary of Prop[3] 

Proposition 5. Any QMR of a UCPS is again a UCPS. 

Let (£, D{C)) and (£&, D{£&)) be the generators of £ and £#, respectively. For the 
following results we suppose that the Ass. 1 is fulfilled. 

Proposition 6. The generators £ and £# are related as follows 

£{u*oF) =C*u*oF,Vu* G D(C*) (11) 

Theorem 1. For all A* G B*(X#) and for allt>0 we have 

pf(Fx,A*)= Pt (x,F- 1 (A*)) (12) 

where (pf) and (pt) are the transition functions of M^ and M, respectively. 

Proof. Let F* be defined as F* : B b {X#) -> B b {X); F*u* = u* oF. Then (Qj) 
becomes (£oF#)u# — (F# o£#)u#, Vw# 6 D(£#) (**). For a strong Markov pro- 
cess, the opus of the kernel operator is the inverse operator of the infinitesimal generator 
of the process. Now, from (**) we get a similar relation between the kernel operators V 
and V* of the processes M and M#, i.e. F* o V* = V o F# on B b (X#), or 

V*u*oF = V(u*oF),Vu* eB b (X*) (13) 

since if u# G B b (X#) thenV^u^ G D(£#).Foru# — 1^# (the indicator function of 
A#), by the kernel operator integral definition, we obtain (fT2l l. ■ 

Relation ( fl3l ) implies the following corollary: 

Corollary 1. The semigroups (P t ) and (Pt) of M^ and M are related by 

P*u* o F = P t (u* o F),\/u* £B b (X*). (14) 

4.2 Adaptive Bisimulation 

In this subsection we define a new concept of adaptive bisimulation for cps. This con- 
cept is defined as measurable relation, which induces equivalent BLFEs on the quotient 
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spaces. In defining adaptive bisimulation, we do not impose the equivalence of the quo- 
tient processes, which might not have Markovian properties [13|, but we impose the 
equivalence of the QMRs (that can differ from the quotient processes) associated with 
the induced BLFEs, with respect to the projection maps. 

Let (X, B{X)) and (Y, B(Y)) be Lusin spaces and let K C X x Y be a relation 
such that II 1 (1Z) = X and II 2 (TV) = Y. We define the equivalence relation on X 
that is induced by the relation 1Z C X x Y, as the transitive closure of {(x, x')\3y s.t. 
(x, y) £ 1Z and (x' ; y) £ It}. Analogously, the induced (by H) equivalence relation on 

Y can be defined. We write Xj-ji and Y/n for the sets of equivalence classes of X and 

Y induced by H. We denote the equivalence class of x £ X by [x\. Let 

B*{X) = B(X) n {A C X\ if x £ A and [x] = [x'] then a;' £ A} 

be the collection of all Borel sets, in which any equivalence class of X is either totally 
contained or totally not contained. It can be checked that B#(X) is a c-algebra. Let 
ttx ■ X — ► X/-ji be the mapping that maps each i£lto its equivalence class and let 

B(X/ n ) = {Ac X/ u \-k- x \A) e B*{X)}. 

Then (X/k,B(X/-ji)), which is a measurable space, is called the quotient space of 
X w.r.t. 1Z. The quotient space of Y w.r.t. 1Z is defined in a similar way. We define a 
bijective mapping ip : X/-R — > Y/tj as 

^([x]) = [j/] if (x, y) £ 1Z for some x 6 [x] and some t/6 [j/]. 

We say that the relation 72. is measurable if X and Y if for all A £ B(X/n) we have 
V>(^4) £ B(Y/tz) and vice versa, i.e. x/j is a homeomorphism. Then the real measurable 
functions defined on X/7J can be identified with those defined on Y/-R. through the 

homeomorphism ip. We can write B b (X/-ji) = B b (Y/ii). Moreover, these functions 
can be thought of as real functions defined on X or Y measurable w.r.t. B#(X) or 
B*{Y). 

Assumption 2. Suppose that Xj-ji and Y/-ji with the topologies induced by projection 
mappings are Lusin spaces. 

Suppose we have given two right Markov processes M and W with the state spaces X 
and Y . Assume that /1 (resp. v) is a stationary measure of the process M (resp. W). 
Let n/n (resp. v/n) the image of /1 (resp. v) under ttx (resp. wy)- Let £ (resp. J 7 ) 
the quasi-regular BLFE corresponding to M (resp. W). The equivalence between the 
induced processes can be used to define a new bisimulation between Markov processes, 
as follows. 

Definition 3. Under assumptions 1 and 2, a measurable relation 1Z C X x Y is a 
bisimulation between M and W if the mappings -Kx and ixy define the same induced 
BLFE on L 2 (X/-ji, n/n) and L 2 (Y/-r, v/-r), respectively. 

This definition states that M and W are bisimilar if £/ti = J 7 /n- Here, £ J-r, (resp. 
T Itz) is the induced BLFE of £ (resp. T) under the mapping ttx (resp. 7ry). Clearly, 
this can be possible iff [ij-ji = v/n. 
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Assumption 3. Suppose that £/-r and T '/■% are quasi-regular BLFE. 

Let M/iz (resp. W/u) be the Markov process associated to £/tz (resp. F/n). 

Proposition 7. Under assumptions 1, 2 and 3, M and W are stochastic bisimilar under 
1Z iff their QMRs M/tz and W/tz with respect to 7Tx and, respectively ny are p/-R- 
equivalent. 

Let U and V be two UCPSs, with the semantics M and W, strong Markov processes 
defined on the state spaces (X, B(X)) and (Y, B(Y)), respectively. 

Definition 4. U and U' are called bisimilar if there exist a bisimulation relation under 
which their semantics M and W are bisimilar. 

5 Applications in Renewable Energy 

Typically, the behaviour of a power system is described by a complex combination of 
continuous and discrete dynamics, where the discrete transitions are usually given by 
a switching controller. Often, these power systems evolve in an uncertain environment 
and their behaviour might be the result of the interaction with this environment. There- 
fore, modeling their dynamics with UCPS seems to be a natural idea. We consider the 
wind turbine description with the associated dynamical equations developed in [2|, but 
we model this in the framework of UCPS. In [2|, the scopewas to compute probability 
distribution function via forward Kolmogorov equation, method that works pretty well 
when the dimension of the state space is small (less than five). When the state space 
dimension is bigger, our methodology to reduce this dimension to one via QMR repre- 
sents a good alternative. In general, a wind turbine model is composed of three subsys- 
tems: the rotor, the generator and the gearbox (that connects the rotor and the generator 
shafts). The system dynamics is modeled as a stochastic hybrid system whose input and 
output are respectively the wind speed v(t) and the generator power Pg(*)- Looking at 
the pictorial representation of a wind turbine from Fig. 1 (also discussed in [ 10|), a sim- 
ple observation shows that the wind speed plays the role a stochastic input for the wind 
turbine. We may model the wind speed as stochastic hybrid physical process and the 
wind turbine as a stochastic hybrid control system. The picture shows that the out of the 
wind speed is the input of wind turbine, so we have to define interconnected stochastic 
hybrid systems. This implies that the study of such a system should consider control, 
simulation, verification problems for the wind turbine model for different modes of the 
wind speed. Of course, this problem has been studied in the deterministic modelling 
framework, but now the main challenge will be to use the right uncertain measures for 
modelling wind influence. 

The relation between driving and braking torques and the turbine acceleration is 
expressed as follows: 

T ■ * drive * brake ,tr\ 

J- L0= (15) 

Ul 

where u is the rotor speed, J is the moment of inertia, Pdrive is the aerodynamic power 
captured by the wind turbine, and Pbrake is the braking power from the generator. 
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Fig. 1. The structure of a wind turbine 



The generator power output is related to the braking power by Pq = r\P\> ra k & , where 
77 is a constant. The aerodynamic power is given by the algebraic relation Pdrive = 
^pR 2 c p (X, 9)v 3 , where p is the air density, R is the rotor radius, 9 is the pitch angle, 
A = Roj/v, c p is the power coefficient. 

The turbine is operated by a switched controller with two discrete modes: 

• In the first mode denoted by q\ (lower to medium wind speed region), the rotor speed 
is controlled by driving Pq (the generator power output) to follow a given optimal 
power schedule S(u>) (see \2\ for the expression of S(ui)) 

• In the second mode denoted by q2 (higher wind speed region), Pq is kept constant 
to its nominal value ?g inom , and aerodynamic power Pdrive is controlled by the pitch 
control in order to keep u> around its nominal value uj nom . 

The controller switches from the mode q\ to mode 52 when uo = uo nom , and back to the 
mode q\ when ui — uo 12 < uo nom . The strict inequality defines a hysteric behaviour that 
is necessary in order to prevent Zeno phenomena between the two modes. 
In the second mode, for the pitch angle is used a proportional controller: 




if 9 = and u < oj r 

if (9 

otherwise 



(16) 



where h{9) = min(6> max , max(6> min , 9 ). 

Following [2 1, we model the horizontal wind speed as a one-dimensional stochastic 
differential equation (SDE): 



dv(t) = 



v(t) 



T 



-dt + kvy/2/fdW(t) 



(17) 



where T — L/v, with L the wind turbulence length scale. The SDE dTvT ) defines a one- 
dimensional diffusion process. A reflecting barrier v — is introduced to preserve the 
positivity of this process. 

The evolution of this wind turbine can be described by a UCPS U whose continu- 
ous behaviour is described by x t — (u>(t),9(t),v(t)), which obeys to the equations 
dT51 >. (fT6l l and ( fTTI l. and the discrete behaviour is described by (q t ) that evolves in 
Q = {ciiiOs)- in this case, the reset map R moves backward and forward between 
the modes qi and q^, without affecting the continuous state. Therefore, the realization 
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of the UCPS U associated to this wind turbine is nothing else but switching process (in 
this case, a switching dynamical system combined with a switching diffusion process). 
For a standard diffusion process defined on X = R" and F = \ ■ |, the Euclidean 
norm (which is clearly a proper map), we consider the corresponding BLFE form: 



Tb n r\ r\ 

£(u,v)=Y] / aij(y)— u(y)— v 



(y)m{y)dy 



u,v G C °°(R"),me C 1 (R n )r\L 1 (R n ;dy) and m(y) > 0, Vy e R", / R „ m(y)dy = 1 
Vi,j : ay = Ojj G Lj oc (M. n ; ^(dy)) where ii(dy) = m(y)dy, 36 > such that 

Vy £R" 

n 

E%(!/)«i>Cveer. 

Then the QMR of this standard diffusion process is given by the following induced form 

/>oo 

e*(u*,v*)= f(r)(u*y(r)(v*)'(r)dr,u*,v* £ D(£*), (18) 



where a.e. r > 0, /(r) = f , =1 ^™.- =1 aij(r9)9idjm(r9)da(9)r n 1 , ct is the hyper- 
surface measure of {y G R n : |j/| = 1}. 

Considering the fact that the evolution of the wind turbine is nothing else but a 
switching diffusion process, its QMR will be given by a "switching" induced form 
which is given by the concatenation of some induced forms whose expressions are ob- 
tained instantiating the form ( TTSl l. 

6 Final Remarks 

In this paper, we have used the concept of energy, which is a key ingredient of 
Hilbertean formal methods, to define QMR and behavioral equivalence for cyber- 
physical systems operating in random environments. Energy is a versatile analytical 
concept that characterizes in a subtle way the interaction between computation and 
physics, as well as their co-evolution. 

Adaptive bisimulation means the energy preservation of the stochastic processes gen- 
erated by the cyber-physical system evolutions. The energy concept can be also used to 
define QMRs for cyber-physical systems. Given a QMR function that reduces the state 
space, we have defined a standard construction that associates a QMR (called standard) 
on the reduced state space. The mathematical results from Section 4.1 show that the 
QMR method preserves important analytic properties (related to HFM). Two uncertain 
CPS are adaptive bisimilar if they have the same energy. The theorem from Section 
4.2 shows that two uncertain CPS are adaptive bisimilar iff their standard QMRs are 
equivalent as Markov processes. 

We have formulated the stochastic model checking problem (a subproblem of 
stochastic reachability analysis, corresponding to the probabilistic model checking of 
Markov chains). We proved two results that show that the problem is solvable for uncer- 
tain cyber-physical systems. The mathematical results from Section 3 provide a upper 
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bound for the reach set probabilities. In this way, one can prove that the probability of 
reaching a state in a certain set can be small enough. 

In renewable energy industry, it is inherent to use multiple energy sources and then 
defining a highly heterogeneous global system. Therefore, networked stochastic hybrid 
system models would be required to model such systems. Then optimization of the pro- 
duced energy should be studied for these networks. That means the switching from one 
system to another one has to be made smoothly with no big "jumps" or big fluctuation 
intervals. Formally, these networks will be modelled by hierarchical UCPS, and then 
for optimization purpose we need to define and study hierarchical control. 

A preliminary version of this paper was reported in Q. 
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